Migrating a Proxmox Forbidden Router

Background

I have been running a forbidden router¹ for some time on a SYS-5019A-FTN4 for some time.
I primarily use it to run my non-storage based services, but now I am gradually hitting its limits and I want to slowly transition my LAN to utilise 10Gb.
So I purchased a Minisforum MS-01.
Now the issue is that I now need to migrate my services to the new machine while targetting minimal downtime and configuration changes (primarily to minimise ip changes).

OPNSense Migration

This was the painful one.
I had a complex to migrate setup with a passed through physical NIC, VPN solutions and VLANS that rely on the physical NIC.

Let me lay out the plan first: I would switch from passing through a physical NIC to using virtual bridges.
There would be two virtual bridges, one for the WAN interface (RJ45 1GB) and one for LAN (SFP+ 10GB).
To achieve this I also wanted to migrate the VM image itself instead of only migrating the config as that would minimise downtime.

Process

With that said, let me go through the steps I did to migrate it.

1. Create a backup file of the VM
   

   vzdump 101 --mode stop --compress zstd --storage local

2. Copy the backup file from the old proxmox machine to the new one
   

   scp root@OLD-IP:/var/lib/vz/dump/* /var/lib/vz/dump/

3. Copy the backup file from the old proxmox machine to the new one
   

   qmrestore /var/lib/vz/dump/BACKUP_FILE_NAME.vma.zst 101 --storage local-lvm

4. Set up the Bridges by modifying /etc/network/interfaces to include

   auto enp87s0 iface
   eno1 enp87s0 manual
   
   auto vmbr1
   iface vmbr1 inet manual
     bridge-ports enp87s0 
     bridge-stp off
     bridge-fd 0
   
   auto enp2s0f1np1 iface
   eno1 enp2s0f1np1 manual
   
   auto vmbr2
   iface vmbr2 inet manual
     bridge-ports enp2s0f1np1 
     bridge-stp off
     bridge-fd 0

   Note: enp2s0f1np1 is the 10Gb SFP and enp87s0 is the 1Gb RJ45
5. Assign the bridges to the VM

   qm set 101 -net0 virtio,bridge=vmbr1 # WAN
   qm set 101 -net1 virtio,bridge=vmbr2 # LAN

6. Then I removed the physical passthrough NIC via the Proxmox Hardware settings for the VM and finally could boot OPNSense.
   I first attempted to assign interfaces via Option 1. (Assign Interfaces) but that erased a bunch of my configs and just was difficult to get working.
   So instead choose Option 8. (Shell) and modify /conf/config.xml

   vi /conf/config.xml

   and then scroll down until you get to interfaces and change the lan interface to the new network interface (in my case vtnet1)

   <interfaces>
     <lan>
       <if>vtnet1></if>
       <descr>LAN</descr>
       ....

   Then shut down your original OPNSense and reboot OPNSense.
7. OPNSense should now have picked up the new LAN configuration and the Web interface should now be available.
   From there you can fix your WAN interfaces. I won't provide the steps for this as my WAN setup is likely quite different from standard to talk with my ISP network.

Now that the migration is complete, you can delete the backup from both proxmox machines as it is not needed anymore

Pihole migration

My particular pihole setup is using a LXC

Process

I tried several times to restore from a VM backup file but it failed due to an issue with permissions and spacec.
Since I have experience in migrating pihole between machines, I knew it was likely much easier to restore from a config, as generally PiHole has much less to configure.
(Note: It may be more difficult if you use pihole with unbound as a recursive DNS, I used OPNSense's built in Unbound DNS server instead. )

1. Create a new Pihole instance, I use the proxmox community script for this
   

   https://community-scripts.github.io/ProxmoxVE/scripts?id=pihole

2. OPTIONAL: Static IP
   I prefer my pihole to use static IPs so I use the advanced setup and randomly assign a static ip as I will change it later.
3. Log in to your old pihole instance and export your config in Settings>Teleporter
   Then log into the web interface of your new pihole instance and import your config in Settings>Teleporter
4. Now we are ready to make the new pihole take over from the older pihole instance.
   Modify eth0 in /etc/network/interfaceswith your old ip address and save.

   nano /etc/network/interfaces

5. Then restart your network stack to use the new IP

   sudo ifdown eth0 && sudo ifup eth0

6. finally shut down your old pihole instance to prevent a ip clash.

haos migration

The simplest migration of all.

Process

1. Create a manual backup in System>Backups>Backup Now
2. Download your backup
3. Create a new HAOS instance, personally I use the community proxmox script for this

   bash -c "$(curl -fsSL https://raw.githubusercontent.com/community-scripts/ProxmoxVE/main/vm/haos-vm.sh)"

4. When you boot up HAOS, choose to setup by restoring from a backup.

Conclusion

Migrating most things over using proxmox is relatively simple, and thats a pretty good thing.
I have been using a forbidden router for many years already and it has saved me several times when I needed to do a rollback from a faulty config.
This is the first time it has bitten me back, and that was because I chose to passthrough a physical nic instead of assigning it like in proxmox.
In the future I will probally switch back to a physcial NIC as Minisforum has done a pretty good job with SR-IOV so I can individually pass through a single interface, but for now I am satisfied.