Using a second firewall

User Support General
1

Hi,

As explained in another thread I have created a minimal sys-wall along with a sys-dns using the structure explained in the documentation:

sys-net <--> sys-firewall <--> sys-dns <--> sys-wall <--> [client qubes]

Everything works as expected.

A problem arises when I add restricting rules for client qubes using the standard procedure: in the firewall tab of the qube I set the proper host, protocol and port (e.g. mybank.com and TCP port 443). Unlike the standard case with one firewall only (sys-firewall) the result is: networking stops to work in the client qube. If I remove the restricting firewall rules of the qube everything works again. Of course, I can manually add custom firewall rules directly in sys-wall (and its template), however I would like to have the easiness of using the GUI. I couldn’t find any information on how exactly rules set through GUI (or through qvm-firewall) interact with underlying firewall software. When I look at the output of iptables -L -n -v and nft list ruleset in sys-firewall I don’t see any of the IP addresses I use in the restricting rules, as if there is no interaction between what I set and the actual firewall.

My question is:

What is the right way to use restricting qvm-firewall rules with the layout recommended in the docs (with a second sys-wall)?

2

Do you really mean that networking stops working, or is just DNS that
stops?
Simple test would be to target by IP address - e.g. http://185.199.109.153

If you have set restrictions for the client qubes, you would not expect
to see those in sys-firewall - the nftables entries will be in sys-wall.
You can see them using nft list table qubes-firewall
When you use the GUI or qvm-firewall for a qube, the necessary rules are created on
the upstream qube to which that qube is attached.

I never presume to speak for the Qubes team. When I comment in the Forum or in the mailing lists I speak for myself.
1 Like
3

Sorry for the confusion. It is not that networking stops but DNS resolving doesn’t work in the client qube.

If you have set restrictions for the client qubes, you would not expect
to see those in sys-firewall - the nftables entries will be in sys-wall.
You can see them using nft list table qubes-firewall
When you use the GUI or qvm-firewall for a qube, the necessary rules are created on
the upstream qube to which that qube is attached.

That’s the info I was looking for. Thanks.

So, I have added the following to my sys-wall which fixes the issue.

/usr/sbin/nft insert rule qubes-firewall forward position 5 iifname "vif*" ip daddr <sys-wall's-IP-address> meta l4proto {tcp, udp} th dport {53} accept

Before I update the guide, I have two more related questions:

Why is this mixed usage of legacy iptables and modern nftables in Qubes OS? Is it deliberate (why?) or is there some work in progress towards removing iptables completely? I can’t find any issue on GitHub or anything in the docs related to that.

Along these lines, is there a way (and would it be appropriate) to use only nftables when one creates custom infrastructure like the one mentioned here? I am asking because it seems impossible to create a sys-wall without installing qubes-core-agent-networking which itself installs both legacy iptables and nftables, supposedly necessary for proper functioning.

This is all very confusing and looks unnecessarily complicated for which I have shared my concerns several times, so I really hope you can clarify the subject.

4

There is no mixed usage.
On nearly all modern systems, iptables is not legacy - it is a
frontend to nftables kernel API, using the old iptables packet matching
code.
You can confirm this by iptables -V which will return the version
number and nf_tables.
When you use this version of iptables, you can use the old syntax and it
will be remodelled to using nftables in the kernel. Any rules you write
using iptables-nft will appear in nft list

In a sense, it’s nftables all the way down.
I advocate using nft wherever you can, although in almost all cases you
could use the old iptables form.

I am sure there is a GitHub issue on this. It’s just not a priority as
long as iptables is functional.

I never presume to speak for the Qubes team. When I comment in the Forum or in the mailing lists I speak for myself.
5

There is no mixed usage.

There is. I explain below.

On nearly all modern systems, iptables is not legacy - it is a
frontend to nftables kernel API, using the old iptables packet matching
code.

I am not an expert knowing many systems but regarding Linux distros in particular - AFAIK most of them use firewalld (frontend) which itself uses nftables as default backend. Anyway, the question is about Qubes OS.

You can confirm this by iptables -V which will return the version
number and nf_tables.

That’s true in the default Fedora template in which iptables-nft is installed. However, consider the minimal Fedora 37 template. To create a firewall qube based on it, as per docs, one must install qubes-core-agent-networking which installs as dependencies iptables-legacy and nftables separately. It does not install iptables-nft as a dependency. So, based on this, the rule translation you mention simply doesn’t happen and the result is - a mix. Does it mean there is an issue with qubes-core-agent-networking which must be reported?

When you use this version of iptables, you can use the old syntax and it
will be remodelled to using nftables in the kernel. Any rules you write
using iptables-nft will appear in nft list

No. That’s not what happens because of the reasons explained above.

What happens is:

In a minimal qube with qubes-core-agent-networking installed, I can write whatever iptables rules and they are visible only in iptables -L. They are not automatically translated and added in nftables rules, and do not appear in nft list ruleset. The result is a mix of unrelated rule sets - one using iptables and one using nftables. That’s why I say there is a mixed usage.

In a sense, it’s nftables all the way down.
I advocate using nft wherever you can, although in almost all cases you
could use the old iptables form.

The problem is that in the described mix an iptables rule can conflict (e.g. cancel or replace) an nftables one (and vice versa). As it is not documented how all this works (which one is first, which one has priority, how qvm-firewall handles things underneath, etc), it is unclear what the final result may be (might have security consequences). Hence the whole confusion and my questions.

I am sure there is a GitHub issue on this. It’s just not a priority as
long as iptables is functional.

I hope you can share a link to it, as I could not find it.

After these clarifications, could you please reconsider the answer?

6

You’ve hit a problem because you are mixing iptables-legacy and nftables.
The full templates don’t have this issue because they use iptables-nft.
You should do the same.
Just install iptables-nft, then install core-agent-networking, and all
will be well.

I don’t know what the web search throws up on GitHub but gh takes me
directly to this issue
There’s also an open PR in qubes-core-agent-linux to resolve this for good

I never presume to speak for the Qubes team. When I comment in the Forum or in the mailing lists I speak for myself.
7

Thanks for the info and links.

As for:

You’ve hit a problem because you are mixing iptables-legacy and nftables.
The full templates don’t have this issue because they use iptables-nft.
You should do the same.
Just install iptables-nft, then install core-agent-networking, and all
will be well.

I am not mixing anything (at least not deliberately). I am simply following the documentation and it does make the clarifications you make, and that results in the confusing mix which I simply noticed. Could you, as an expert, probably correct the documentation? (firewall, minimal templates)

8

Correcting myself:

“it does NOT make the clarifications you make”