HVM qube does not have internet when gateway ip is set to the ip specified in qube settings but does have internet when gateway ip is a random ip?

I have a standalone hvm LinuxMintDE 6 “Faye” install similar to your Kali install which I occasionally use for testing some things. I also have a standalone sys-vpn pvh vm running Fedora 40 and mullvad vpn. AppVMs pointing to the sys-vpn work great out-of-the-box (OOB). So I too was a bit puzzled when experiencing the similar behavior. I ended up doing various tests trying to determine what was going on. For what it’s worth, I thought I might share the results. Maybe someone with Qubes technical knowledge can provide internal details.

There is a great forum explanation for HVM networking titled “Setting up Networking for HVMs” on this page https://www.qubes-os.org/doc/standalones-and-hvms/. However, in this article it says to copy the GW also so I don’t know if the behavior seen is a bug that got introduced after that was written, nor why it doesn’t presently work to do so. FYI, I’m on Qubes 4.2.3.

Here is the short version of what I found in testing. (Read this if you want results and don’t want any details.)
In short,

  1. QubesManager’s guest GW setting seems to circumvent or disregard the hvm guest NIC GW settings that from the instructions say to copy into the guest NIC GW settings. And putting the QubesManager hvm guest default gateway setting in the guest’s NIC GW setting breaks the internet access.
  2. Qubes’ IP and DNS IP addresses need to be mirrored into the guest NIC settings. (see the aforementioned article).

Here is the long version. (A more detailed explanation of test results. Sorry about the length… :grimacing:)
When creating a new vm, Qubes sets up the guest IP addressing to tie the vm into the Qubes environment. By setting the “netVM” on this guest to the sys-vpn qube so that you can route through sys-vpn, Qubes automatically puts the sys-vpn IP address into the QubeManager “guest” default gateway (GW) setting. Again, this is a setup that works OOB for appVMs.

Because it was necessary as part of the hvm install to mirror the NIC settings from the QubesManager, I also logically mirrored the IP of sys-vpn into the default GW in the guest nic settings and this is what the instructions say to do. The instructions said “In order to manually configure networking in a qube, one should first find out the IP/netmask/gateway assigned to the particular qube by Qubes. This can be seen, e.g., in the Qube Manager in the qube’s properties:”. I thought “No problem”, if it followed appVM behavior, this should work. However, that was a mistake I later found and the result was no internet. Rechecking the guest nic config showed the nic interface was connected. It looked like a routing issue.

Further testing determined Qubes doesn’t seem to even use the value of the hvm guest NIC GW setting EXCEPT maybe as a placeholder. What I mean by this is you can put just about anything in there as long as it’s NOT the actual GW listed in the QubesManager default GW setting (doing so breaks internet access) and Qubes doesn’t care it’s not an actual valid GW or route. It also doesn’t like an empty GW of course or something like 0.0.0.0 which is the same as empty. Well experiencing this behavior just doesn’t seem right but I can get internet access and verify it is using sys-vpn using just about any gateway when I manually configure the hvm guest NIC GW with any of the following:

  1. with my home router GW IP;
  2. an IP of either of the 2 Qubes DNS servers;
  3. the IP of the GW of sys-vpn (is not the IP of sys-vpn but rather sys-vpn’s netVM);
  4. any random private IP address ie. 172.10.10.10;
  5. any made-up IP address ie. 200.200.200.200, or 1.1.1.1.

This leaves at least the perception that any of these options don’t do anything in that field; they seem to have no effect. And these aren’t even gateways to sys-vpn. Yet the traffic apparently routes there and continues through the vpn on out. Again I verified each of the examples as being put in the NIC default GW and heading out to the Mullvad site where it clearly showed I was using the vpn. In other words, it appears the route to sys-vpn then is set purely with the QubeManager default GW entry for the guest which of course in this case is the actual IP of sys-vpn (or netVM of the guest). This would be the expected behavior especially if I used it in the guest NIC GW setting. The fact that it still gets set correctly isn’t an issue because I would expect Qubes to set that route correctly. But what kind of is not so expected is that putting that IP into the NIC GW kills the internet.

By Qubes setting this route, Qubes DNS also now appears to be available to the hvm guest provided that:

  1. you use a valid placeholder; and
  2. provided you also mirror the Qubes DNS IP’s in the NIC like the instructions say.

For example, if I put a Google DNS server IP 8.8.8.8 in the NIC DNS settings in the same situation, there is no DNS resolution and no internet even though I have a valid “placeholder” set in the NIC GW. Only when I put the Qubes DNS servers in does internet resume working. I would expect that behavior to prevent attempted DNS workarounds. What I’m saying is that all the “correct” Qubes pieces need to be used. Well it seems except for the NIC GW.

To summarize:

  1. using any one of the above 5 scenarios as a choice in the guest NIC GW setting doesn’t seem to prevent the QubesManager guest GW setting from working correctly even if they’re not actually gateways to sys-vpn. Qubes just seems to ignore them. The positive side of this is that Qubes is maintaining control of the GW.
  2. Having an empty NIC GW setting to me would be an expected failure and it does.
  3. But trying to use the guest GW setting from the QubeManager in your guest’s NIC GW setting isn’t allowed. Doing so breaks internet access. This is odd behavior since that is the true GW to sys-vpn. And Qubes maps it to it no matter. Lastly, the instructions support doing so.

So my take on this behavior is possibly a bug, or this is probably by design (a new feature?) and maybe for security reasons so users can’t accidentally break the security or maybe the routing within the guest? This remote possibility is reinforced by the need to mirror the “Qubes determined” networking so the guest vm is using the Qubes environment and not circumventing it. So I suppose…

Well I’ll finally end :sleeping: by saying it doesn’t look right, but you can get it to still do what your goal is which in my case was to send hvm guest internet bound traffic through a vm hosting my vpn; the same as I can do OOB with an standard appVM. And I can verify it is actually using the vpn. :grinning: My apologies for the length…