Proposition for a simpler networking model - would it work?

I’m still learning Qubes, so please forgive me if I misunderstood some concepts.
The idea is to use bridging instead of routing (kind of). I know that routing is preferred to bridging to prevent layer2 discovery on dom0, but what do you think about those mitigations, are they good enough ? Also, my method kind of prevents those kind of escapes.

To explain my idea, I’ll show how I did it in my Qubes-like setup, I’m too noob at Qubes to compare.
Note: below, when I say “client”, I mean any domain that needs to access network ressources, so dom0 or any domU.

My router/firewall is a domU, protecting all other domains, including dom0. It has all network cards (NIC) passthrough’ed from dom0, like in Qubes.

  • On dom0, I create a bridge per client. So the bridges have no linked NIC. No IP either (except the one for dom0 as a client).
  • In the firewall, I create as many VIFs that I have clients.
  • all the domUs are created with a VIF on their corresponding bridge.

Example: bridge brdom1 hosts two VIFs, fwdom1 and nicdom1.
So if dom1 wants to access dom2, it’s done like :

nicdom1 -> ( fwdom1 - fwdom2 ) -> nicdom2
# the parenthesis represent the firewall domain

The advantage of this method is config centralization, you don’t need to have an additional firewall on the domains (although you can), as they’re alone in their network segment, and cannot escape from the bridge.

If not mistaken, my firewall does the job of sys-net and sys-fw. But couldn’t this setup be slightly adapted to Qubes ? Of course, any distro would work as a firewall, I’m more talking about the principle behind.

Any feedback welcome !

Drawback to my setup

Because “max_vifs=8” for a Xen domU, if there’s more clients than that, it needs additional setup :

  • either setting VLANs on both the domU and firewall VIFs
  • or chaining several firewalls

I’m probably missing something here.

Dom0 has no networking at all. Its an island. There is no address or virtual nic associated with it so its not reachable or discoverable on any networking layer. How is this layer2 discovery going to happen unless the intruder already has control of Dom0, and adds in that networking just so they can attack it? If so then you would have much bigger problems on your hands than network discovery.

It is my understanding that a bridge just takes what is on one side and dumps it on the other side, and that is dangerous unless you put in lots of additional filtering which takes additional processing power. Part of the design of Qubes is to specifically limit AppVMs from even seeing other peer AppVM’s so using a bridge for a network would completely remove that level of protection. Sys-firewall prevents that right out of the box unless you build lots of rules to permit it.

sys-firewall already provides filtering in one centralized place, though I have at times exceeded the number of rules that it can handle.

Yes, but it’s on me !
I used -my- setup to show how it could be done with a layer of iptables less because I just started Qubes, and didn’t want to fail trying to explain it the Qubes way.

I don’t have network working in Qubes yet due to the way I’m using it (nested as a domU), so no sys-net, no sys-firewall. Maybe you could explain to me how is the networking done ? I mean, under the hood. Also a redacted ip a from sys-net and sys-fw would really help.

I’ve never used routed networking in Xen, only bridging, but -my- way : the usual use case when bridging on Xen is to put ALL domU vifs on the physical NIC (like on Virtualbox if you want, but my info is 4+ years old), and I really don’t like that, because of the reasons explained in the link in my 1st post.

Here is how it would be done :

  • physical NICs stay in sys-net, or any netVM
  • the bridges would be created on sys-firewall, but one bridge FOR EACH appVM
  • so on those bridges, there are 2 vifs : 1 for sys-fw, 1 for the appVM
  • the vif of sys-fw on a bridge acts as the default gateway for the vif of the appVM, so in the end, each appVM is in its own network segment, shared with none of the other

The reasoning behind this is to alleviate the firewalling management to a dedicated firewall distro (with or w/o a GUI), so :

  1. Qubes dev have less work
  2. if it’s a GUI manageable distro, users have a GUI to manage it, and don’t have to mess with iptables rules (I know rules have a GUI in Qmanager, but see point 1 ^^)

PS: I’m using one pfSense to firewall dom0 and all my domUs, but really any distro would work this way

PS: I’m using one pfSense to firewall dom0 and all my domUs, but really any distro would work this way

Dom0 is not supposed to have a network because that makes the whole system hackable/vulnerable. The security is through hardware separation, so connecting what is supposed to be separate is kind of going backwards.

As far as pfSense replacing sys-firewall that part is quite possible as many other people have explored that with other firewall software. Personally I use pfSense on my physical network and while it might be fun to try as an internal firewall that is a lot of work for something that has marginal benefit.

firewall management - I really don’t see any added benefit with bridging verses the stock managed routing. The basic routing is set up correctly right out of the box and you can then punch holes in the firewall using rules if you really need something different. It sounds to me that you are just punting the CPU loading up into the Xen system by adding so many virtual bridges. Either way the CPU is going to spin and do work and then have nearly the same result from the users perspective.

Just for security reasons I tend to move away from bridging unless is solves an actual problem. Bridging is designed to let everyone communicate with everyone else by default, which is the opposite of the Qubes paradigm. I realize that here you propose a one to one bridge to each individual AppVM but that is just making more complexity with the same basic result as the default Qubes system out of the box.

I know, this is just how it works on my setup right now (though it has no external access and no access to physical NICs), I’m clearly not arguing to put some network stuff in Qubes dom0 ! ;)
My actual dom0 hosts network services for my local network (NAS, DNS, mail, etc), which I would not do on Qubes, AFAIK that’s not recommended ?
Although I want to move my user domUs to Qubes, which itself would be hosted by my actual dom0.
I only have one powerful system ! And even if paranoid, not a threat model that high.

I’ve quickly seen MirageOSfirewall has been tested, right ?
It’s just I find it way easier to manage rules via a GUI than having to edit an iptables file (I use both on my network). I’m thinking about the average user who has to understand how to write correct iptables rules (not giant holes), remembering the IPs or the network ports used by some protocol (aliases on pf), that there are 2 firewalls (one on appVM and one on sys-fw), etc. Plus it could remove the need to manage the firewall handling code in Qubes ?
But I may be completely wrong, or I’ve read the wrong guides, or maybe they’re not up-to-date (I’m testing 4.1) ?
On the other hand, a lot of work ? I don’t see how as pf supports PV drivers. But that may show how few I understand the Qubes way.

Performance-wise, I admit I have no idea, even if it works here quite well with 120MB/s DLs going through pf to a domU which then stores it on a TrueNAS.

I think I need to play more with Qubes before arguing more (if it’s not pointless ? ^^), but w/o networking that’s hard.
I was also hoping to learn, by technical comparison, more about how Qubes works under the hood with that proposition.