Unable to shutdown Mirage-firewall without netvm attached

Mirage-firewall vm was installed on a fully updated Qubes installation, with the latest mirage-firewall tar.

When I have sys-net connected as mirage-firewalls Netvm, everything works as expected. When I remove the net-vm, mirage-firewall refuses to shutdown, and qubes with mirage-firewall as their netvm will not start either. Eventually I get a libxenlight error after time from initiation of shutdown.

The errors from logs I get (I can get more on request) are:

In /var/log/libvirt/libxl/libxl-driver.log

libxl_domain.c:853:pvcontrol_cb: guest didn’t acknowledge control request: -9

When I grep mirage-firewall in journalctl:

unhandled exception while calling src=b’dom0’ meth=b ‘admin.vm.shutdown’ dest=b’mirage-firewall’ arg=b’’ len(untrusted-payload)=0

If I attach a VM with mirage-firewall as its net-vm when mirage-firewall is up but has no net-vm itself, I get:

Start Failed: internal error: libxenlight failed to create new domain ‘openbsd-21-sysnet’, see /var/log/libvirt/libx/libxl-driver.log for details.

In libxl-driver.log:

libxl_device.c:1146:device_backend_callback: Domain 25:unable to add device with path /local/domain/21/backend/vif/25/0
libxl_dm.c:2761:stubdom_pvqemu_cb: Domain 24:error connecting nics devices: Resources temporarily unavailable
libxl_dm.c:2800:stubdom_xswait_cb: Domain 24:(null): startup timed out
libxl_create.c:1913:domcreate_devmodel_started: Domain 24:device model did not start: -9
libxl_device.c:1434Llibxl_wait_for_backend: Backend /local/domain/0/backend/pci/25/0 not ready
libxl_device.c:1146Ldevice_backend_callback: Domain 25:unable to remove device with path /local/domain/23/backend/vif/25/0
libxl_domain.c:1553Ldevices_destroy_cbL Domain 25Llibxl_devices_destroy failed

The reason I want to have mirage-firewall run without an assigned net-vm is that I want it to work with an OpenBSD netvm, and so it must have no netvm attached to it.

No devices are attached to the mirage-firewall VM.

What might the issue be here?

Could the issue be that sys-net was set as a --property netvm= during the installation of mirage-firewall, and so it refuses to operate without it?

@Unman Have you tested mirage-firewall with OpenBSD?

I get the above errors booting without a netvm assigned, but when I set the Personal-VM to 'provides network=true" and remove its netvm, then attached Personal as mirage-firewalls netvm, it boots and shuts down as expected.

Is there a work around I can use here that doesn’t break OpenBSD’s model?

Edit- is the work around to this just to have:

AppVM → Mirage-firewall → sys-firewall ← openbsd-21-sysnet ?

That should still offer all the reduced attack surface where it matters from mirage firewall and give me openbsd (because sys-firewall can have no netvm attached and run).

If sys-firewall got compromised that wouldn’t jepordise the function of mirage-firewall.

That setup should work and, as you said, provide similar attack surface reduction guarantees, at the cost of an extra VM. The issue that you are having with mirage-firewall is probably due to some issue with IP allocation on the external interface when no netVM is attached. Did you look at the mirage-firewall logs to see if there was something there? It is likely that this use case was never tested by the mirage-firewall developers.