Ok I am kind of lost looking through the qubes firewall files and I would please need some help here
As far as I understand it half of the firewall rules apply from /etc/qubes folder and the other half (qubes-firewall table) from a daemon? Can someone explain further on that please?
Also it seems to be users are supposed to install their firewall rules in /rw/config/qubes-firewall-user-script but when I place my nft file in /rw/config/dns.nft for example and link it inside the qubes-firewall-user-script it doesnt execute (i think years ago i had to do something like $(sleep 8 && /rw/config/iptables.sh) ) In the end I am using /rw/config/rc.local which as far as I know is depricated and will be removed
On the other hand I am not sure what rules I CAN change and what rules I CANNOT, for example I have edited my sys-firewall nftables’s dnat-dns to hijack any dns calls and send them to sys-dns which is directly connected to it, I added forwarding rules in custom-forward to sys-firewall and output and custom-input rules to sys-dns, I have done all that to avoid having to execute a different sys-dns for each of my private vms
What I cannot understand is the inner workings, like does qubes EVER overwrite the nftables after it has already applied them? I suppose it wourld change the downstream and allowed sets but DOES it ever apply changes to these firewall rules that could petentially cause me to leak data in the future? The dns settings are just an example but this is something I want to understand in general, I need to know if now or if there are plans in the future to make firewall
As it stands my understanding is that I should just create a separate table, add my custom chains there, and just make sure my nftables run before the qubes rules to avoid them being overwritten but that creates extra confusion on my part which will just lead in extra mistakes because I will have to constantly scroll back and forth from my rules to the qubes’s standard rules to avoid creating any leaks or at the very least to make sure everything works correctly.
What are the scripts I should study through?
Which folders are most important nftables in qubes for me to pay attention to?
All that and I haven’t even started on how confused I am on if commands like qvm-service can tear down my firewalls or not, so I am not even touching that subject yet.
I understand I have a lot of questions pilled up but I prefer working through my problems and then stacking up questions rather then making a post here and there on every problem but I cannot help it, to my mind it appears that the docs page of qubesos is quite outdated so I can only try to voodoo magic my way out by reading the scripts.
I may be giving you false info. I will tell you what I know so far.
As far as i know, by asking https://perplexity.ai and checking documentation and git repos…
For NFT rules:
/etc/qubes/qubes-*.ntf files (ntf rules) are stored in the template vms, and they are created when installing qubes-core-agent
This rules can be modified by writing into the /rw/config directory of AppVMs to make them persistent. I think it’s also possible to directly change them in /etc/qubes/qubes-*.nft in the TemplateVMs
For the firewall rules:
Installing qubes-core-agent into a qube makes a VM be “Qubes-aware”. qubes-core-agent-networking should be the package responsible of managing networking
The rules, like firewall rules, are stored in dom0. You will find them in /var/lib/qubes/appvms/<qube-name>/firewall.xml
Networking is stored in QubesDB. So dom0 knows where rules have to be applied.
Then, this agent will manage the rules once the VM boots. I think this is done via QubesDB (with its own transport channel). dom0 passes the rules to the qube networking agent and he loads them
When changing the rules, dom0 signals the qubes to reload the new rules (also done via QubesDB). Those qubes get the event and do the reload thing also using NFT.
And about the customizations you are supposed to make to the NFT rules, check in the default templates of /etc/qubes/qubes-ipv4.nft and /etc/qubes/qubes-ipv6.nft the bottom. You will find something like this:
# These chains are reserved for end-users. Qubes OS itself will
# never modify them.
chain custom-input {}
chain custom-forward {}
I think those are the inbound and forward NFT rules you are supposed to change. For outbound traffic, i think it’s supposed to be done inside the firewall.xml (via qvm-firewall).
I also think it’s possible to declare inbound traffic rules in the firewall.xml files, but i haven’t tried it yet.
Do not trust blindly in Perplexity and me and double check with an expert please.
I will continue exploring this. If I get to know this more deeply, I will tell you asap here.
TLDR: TemplateVMs have a default NFT template that can be customized. This customization can be also made in AppVMs. dom0 is responsible to also add NFT rules by querying the firewall.xml files and using QubesDB to do so. qubes-core-agent-networking is the package responsible for networking inside VMs.
I also am unclear if you are intending to work in qubes that provide
qvm-firewall service, or more generally, in other qubes.
You can do this - but it seems to me that much of what you want can be
dealt with only in the custom-input and custom-forward chains - these
chains are guaranteed not to be overwritten.
As always it is much easier to deal with concrete examples, and
address specific use cases.
You do not say how familiar you are with nftables.
Rules in rc.local will be processed after the network is up - probably
not what you want.
But you should be able to use qubes-firewall-user-script or nft
scripts in qubes-firewall.d These scripts need to be executable, of course.
The order of processing is:
/etc/qubes/qubes-firewall.d
/rw/config/qubes-firewall.d
/rw/config/qubes-firewall-user-script
You can also put scripts in /rw/config/network-hooks.d to run when
interfaces come up or down - I use this to overwrite the dnat-dns chain
, combined with a drop rule in custom-forward to stop dns traffic moving
up the network. That seems like something you could use.
But I would begin by making sure that I am able to run any such scripts.
I never presume to speak for the Qubes team.
When I comment in the Forum I speak for myself.
I assume I intend to do both, I not yet too experianced with qubes’s qvm services and qvm-firewall that match, as far as I knew a few years back(around 2022) when i last used qubes is that qvm-firewall does not touch the nftables inside the qubes themselves but does its magic on the dom0 itself
That is what I want to do as well but to be fair I have had a really bad experiance with these folders and scripts even working at all, just as I explained a few years back i had to do something like $(sleep 8 && /rw/config/iptables.sh) for it to work at all, my understanding is there is a race condition is in there and as far as I am conserned race conditions are a no-go, I think I am going to test creating a custom .service that runs After=qubes-firewall.service with a RetrySec=1 but in such a hypothetical case I would have to Exec 2 scripts one flash/delete my custom chains and another to run the chain itself just to avoid repeating rules or erros such as cannot delete chain cuz it doesnt exist
I dont have a master’s degree in networking thats for sure, I was more familiar with iptables and while I triple check everything networking related I make for holes I still want the reasurance that my os itself wont fuck me over, I think it was the reason I switched back from qubes to gentoo with a custom xen setup years ago.
nevertheless while I understand that rc.local is not the proper place to place such a script I still need to understand why a race-condition happens in qubes-firewall-user-script for me to use it properly
Honestly man I feel you, I dropped in the forum because the combonation of venice.ai deepseek, the docs of qubesos forum and the scripts themselves didnt give me acurate information enough that I can lock in
You have not filled the gap I pointed to.
What you say isnt quite right - qvm-firewall is used to set nftables in
the qube’s netvm, not in the qube itself.
Not my experience at all in the time I’ve been using Qubes. Certainly
not with nft, and I customise some qubes heavily.
You dont need a masters. A clear understanding of what you want to do
and familiarity with nftables will help. I hope I’ve given you some
reassurance that it wont be Qubes that shoots you in the foot.
Without knowing what you are trying to do I cant help you with the
claimed race condition.
I never presume to speak for the Qubes team.
When I comment in the Forum I speak for myself.
So I went over the .service files in /etc/systemd/system as well as the scripts the executed and stress-tested my networking as well as looked it over with tshark at multiple points in the chain. I posted my general DOs and DONTs in this link
I hove it helps anyone in need in the future, I wrote down petential leaks and what is the strongest defence against them to this point by my knowledge.TL;DR related to this post, to my supprise you cannot execute .nft files directly inside qubes-firewall-user-script, but you do not have to add a sleep timer either just like I had to do a few years ago, all that is needed is to subshell $(/rw/config/my-firewall.nft)
If anyone needs any more explanations drop a comment here or in the linked post