Opensense on Fujitsu Futro (2)

2021-12-13

As a follow-up to Opnsense on Fujitsu Futro …

Meanwhile I've bought four more Futro thin clients (two S900, two S720) and had luck, because the S900s came with PoE modules (1 Gbit/s) and the modules were compatible with the additional S720s. My original S720 will turn into sort of a headless always-on workstation (i.e., running a kubernetes control node).

Today I finally got around to setting up the two newer Futro S720s as a high-availability pair of opnsense firewalls. There were a few caveats to resolve, but all in all I'm happy with the outcome.

Unlike other vendors, opnsense firewalls in HA are not equally entitled, but act as primary and backup device/node. This means replication of config is a one-way road (primary to backup).

HA functionality of opnsense is provided by

config replication (primary to backup device) and
virtual IP takeovers (via CARP)

Some things I stumbled over along the way:

Config replication (xmlrpc) needs the backup firewall's web UI accessible (from the primary fw), which means there needs to be a firewall rule allowing traffic and the web interface needs to bind to the correct interfaces. Since I had my devices configered to bind the web interface only to the MGMT interface, this meant that all my firewall rules were fine, but nothing responded.
Config backups and recoveries are only compatible across devices, if the internal device names are in sync. This means that the order of interface assignments matters (an interface like internal34 interface (opt4, re0_vlan34) is internally only referred to as opt4), so one needs to take care of matching the optX interfaces across devices.

My setup is:

two physical devices, each with two network interfaces (re0 is onboard, re1 is the PoE module) -- in hindsight, I should have swapped them, because I will most certainly not have a PoE switch for my WAN interfaces, but I might end up with a couple of available PoE ports on the LAN switching side.
several VLANs assigned to re0: hasync, guest, internal, iot, entertainment, …
subinterface hasync as quasi-dedicated interface used for configuration (xmlrpc) and state synchronization (pfsync)
dedicated account hasync
WAN interfaces configured for DHCP, but also with a dedicated CARP interface shared between devices
no automatic outbound NAT -- this was important to make CARP work properly on the WAN interface; according to an opnsense forum thread, DHCP setup for CARP interfaces does not work well