First, check that is actually the case:
$ qvm-pci ls sys-firewall
If there is indeed a controller attached to sys-firewall, detach it:
$ qvm-pci detach sys-firewall [BACKEND:DEVICE_ID]
The [BACKEND:DEVICE_ID] looks something like this: dom0:0x_xx.x
Depending on your case, you may want to attach the extra controller to sys-net:
$ qvm-pci attach sys-net dom0:0x_xx.x