I really do not understand how does network works on qubes.
Have clean setup.
Created default appVM based on debian-10.
Connected directly laptop to fa0/1 of my router and eth0 with my laptop.
Make it same subnet 10.1.1.0/24
On router : SVI - 10.1.1.1/24, fa0/1 in access mode. On Qubes sys-net eth0 - 10.1.1.2/24
From appVM i can ping myself and gateway, but from router can’t. Why?
Everewhere firewall is disabled and clean.
As sys-net and sys-firewall i use clean centos-7 templates. There is no firewalld too.
Never touched iptables/nftables.
Does anyone know how to debug this?
Do I understand correctly that you want to ling your Qubes system from the outside world? Is this just for debugging or do you want to achieve something specific?
You said you never touched iptables but that might just be the problem. Did you have a look at iptables -L -n -v in your sys-net?
I checked for mine and it shows some rules which basically block all external traffic.
I think that by design, Qubes is pretty locked down and even though I don’t know something specific on this, I suppose this could happen automatically if you mark a VM as “provides network”.
Me neither. All I know is it’s a complete mess.
I’m able to ping my Qubes machine from my router. I had followed the port forwarding guide at one point, which involves making changes to iptables, which is probably what resulted in ping now working.
I think that only affects outgoing traffic and NAT. It won’t do any port forwarding or accept incoming traffic without additional configuration as far as I know. Basically, it causes firewall scripts to be invoked whenever a client VM is added to or removed from the NetVM. qubes-firewall-user-script is where you usually make your changes, e.g. for port forwarding. (See Config files | Qubes OS)
I probably should have been more explicit. With “this” I meant the rules in iptables which drop all traffic coming into the physical network interface. I guess these are by default added to anything that you mark as network VM in Qubes to have a failsafe default configuration.
I’ll rise to that.
The intention is to provide a reasonable set of rules that provide
security out of the box for an ordinary (desktop) user. I think the
current position is OK for that.
It’s simple enough to set firewalll rules (with known exceptions),
and they will “follow” the qube if it is assigned to a new netVM. Well
and good.
Complications arise because of the way that Qubes provides network
services to separate qubes.
This means some heavy lifting if you want to expose a qube’s ports to
the outside world.
I think the assumption is that if you want to provide more complicated
set-ups, then you should be able to deal with the firewall, and
understand the security position. Yes, I know there are scripts there
to set-up VPNs and open ports to the outside. The problem is that
if you provide means to make things easy then people will (blindly)
follow them without understanding.
iptables/nftables are not difficult to understand.
Basic network troubleshooting is not difficult to understand.
These are skills that almost anyone can learn and that are useful in
understanding the security position one has taken. There’s nothing Qubes
specific here - I think the same applies to almost any OS.
In the Qubes case, networking is (relatively) simple.
Each netVM provides a firewall (with some basic rules already set), and
provides masquerade function. All traffic from qubes connected to a
netVM appears to come from the “external” IP of the netVM. Traffic
moves through the netVMs until it reaches sys-net, and leaves the Qubes
appearing to come from the sys-net IP.
New inbound traffic is blocked by default, but connected traffic is
allowed back down the chain of netVMs.
That’s it.
And now you can see why it’s somewhat difficult to allow inbound traffic
to a specific qube: not only do you have to open the relevant port on
each firewall, but you must ensure that the traffic is correctly
routed through the connected netVMs until it reaches the destination
qube.
That can be complicated if you have a set-up with many netVMs, but if
you take a methodical approach, it should be done, and you will have
learnt a new (essential) skill.
Of course, there are people who want the benefits without learning the
details - they don’t have time or cant be bothered. OK, for those people
you might prepare a script or set-up that will work for them. That’s
fine. But they are more likely to be at risk than someone who
has taken the time to understand what they are doing.
3 Likes
Thank you for the detailed explanation, unman.
I couldn’t agree more.
Considering that Qubes is designed to keep the user protected from the outside world, it should be very difficult to intentionally drill holes through that shield. If one wants to be able to access VMs from the outside world easily, it’s quite possible that Qubes is simply the wrong tool for the job, IMHO.
1 Like
Every NetVM performs another layer of NAT, too, not just sys-net, right?
If yes, what are the benefits that justify the complexity of using NAT (instead of normal routing and filtering) at each and every hop? I understand why it is used in sys-net, but what is the reason for additional layers of NAT within the borders of Qubes?
Agreed, but there can be another side to this coin too.
This implies that, if port forwarding is a requirement for you – you just need it – and you don’t know enough networking and iptables stuff to do it yourself, then your options are:
- Don’t use Qubes at all, or any other OS where port forwarding is difficult.
- Download a script and follow a tutorial you don’t really understand, and screw around with it until you get it just barely working, and hope it’s secure.
Are either of those more secure than, for example, clicking a few buttons in a simple hypothetical tab in the Qube settings dialog, or using a simple qvm-port-forward command? In other words, are we really doing users a favor by making it difficult?
It’s kind of like saying “schools shouldn’t give out condoms because it encourages kids to have sex.” People are going to do what they’re going to do; you might as well try to make it as safe as you can.
Just another way to think about it, that’s all.
Agreed, but there can be another side to this coin too.
This implies that, if port forwarding is a requirement for you – you just need it – and you don’t know enough networking and iptables stuff to do it yourself, then your options are:
- Don’t use Qubes at all, or any other OS where port forwarding is difficult.
- Download a script and follow a tutorial you don’t really understand, and screw around with it until you get it just barely working, and hope it’s secure.
Are either of those more secure than, for example, clicking a few buttons in a simple hypothetical tab in the Qube settings dialog, or using a simple qvm-port-forward command? In other words, are we really doing users a favor by making it difficult?
Nice straw man.
I have set up many users with Qubes network configurations that I
know they don’t really understand. And that’s fine.
One danger in providing " a few buttons to click" is that some users
just will click those buttons - they dont know what they are doing, and
end up with effects they did not expect. Any one who has worked a help
desk knows that this happens all the time.
It’s kind of like saying “schools shouldn’t give out condoms because it encourages kids to have sex.” People are going to do what they’re going to do; you might as well try to make it as safe as you can.
Just another way to think about it, that’s all.
Actually, I advocate giving the tools and educating users as to how to
use them.
To follow your analogy, it’s handing out condoms and providing
Sex Education.
It’s an established fact that adolescents who do not have formal sex
education are half as likely to use a condom at first intercourse and
even less likely to use condoms consistently.
Seriously, if security is important to you then you need to learn this
stuff.
Yes, MASQUERADE all the way down.
The obvious advantage is that it hugely simplifies the network
configuration.
Think of Qubes as a network-inna-box.
Let’s say you have 3 netvms chained with appVMs attached to the 3rd.
The rules on 1 and 2 are simple, and invariant - just pass traffic up
and down to the next IP hop, (with some filtering). You can fire up many
qubes attached to the 3rd netVM, and those rules aren’t affected: it’s
only the rules on the 3rd NetVM that change and enforce filtering.
In a complex network infrastructure this is “normal routing and
filtering”.
Okay, I think I see what you’re saying. It’s so the next hop’s IP is always the same from the perspective of each of the first two VMs. (Assuming there’s only one downstream NetVM attached to each upstream NetVM, I guess?) This way the first two VMs in the chain don’t have to be reconfigured.
My thinking was that using NAT only in the outermost NetVM, if you’re only using one sys-net, then sys-net would be your one centralized location for all your DNAT rules, and then any other NetVMs in the chain would usually not require any configuration at all in order for port forwarding to work, except probably to open the port in sys-firewall (though in that case you just need to specify the port, not the IP).
I guess there could be advantages to each one, in different scenarios?
Thanks for taking the time to explain.
Well, okay, I meant to say “routing and filtering without NAT”.
Sorry that was actually supposed to be a reply to adw’s post but I only got you in the quote. Either way, it was intended as an alternative viewpoint, not a counter argument.
Well it sounds like we’re in agreement about that then.
In this case I’m just not really sure how taking the time to do the actual DNATing by hand on three different VMs is really going to teach you the security theory and implications of port forwarding. “How” versus “why” (or “why not”). But there probably is a reason, and you’re probably right… like I said, I meant that as a perspective to consider, not as an argument to anything.
I guess what I was trying to say is that, if you make everyone dig down into the iptables and stuff for every port forward, sooner or later one of them is going to accidentally put an accept rule in the input chain instead of the prerouting chain, or misconfigure something in some way that makes it worse than what the correct port forwarding would have. Anyone can make a mistake.
If you provide an easy(ish) and predefined way of safely doing it, that’s giving them the tools to be safer than the aforementioned method, IMO. They still need to understand what port forwarding is, the implications, and so on, but not necessarily the gory details of how to implement it.
To clarify, I don’t think it should be made intentionally difficult (nor do I think it has been). Rather, I think it should be difficult in the sense that the Qubes devs shouldn’t take time away from more important work in order to make it very easy for users to shoot themselves in the feet (in other words, advanced user DIY-level of difficulty).
I wouldn’t be opposed to such a tool (as long as it has good anti-footgun protection). I just think that there’s so much higher-priority work to be done that I can’t see there being enough bandwidth to get to this anytime soon. Maybe if it were a community contribution?