Can someone help me here? I made all the steps correctly. My OpenBSD received internet succesfully and my mirage-firewall worked fine, since i tested a setup which i use default Qubes sys-net > mirage-firewall > AppVMS and it worked. But when i test your network setup (sys-net-openbsd with mirage-firewall net qube, mirage-firewall with n/a and appvm with mirage-firewall net qube), it just doens’t work. When i test commands like ping 8.8.8.8 in a appvm connected to mirage-firewall, it just show “10 packets transmitted, 0 packets received, 100% packet loss”. How can i identify exacly what’s the problem and fix it?
You have succesfully used OpenBSD as sys-net? If yes, can you help me? I stopped here
The “firewall” step i don’t know what i need to do exacly.
On the firewall, you tell mirage where to forward the packets. On that step you need to set netvm for mirage to none and add kernelopts. Then you need to restart it.
1 Like
The “kernelopts” you are talking about is “–ipv4=10.137.x.xx --ipv4-gw=10.137.x.xx”? If so, i already did that. The only difference on the setup that occurred to me its the fact that my installer didn’t ask about my gateway’s ip and the dns nameservers. So, after installing OpenBSD, i set it up manually. I don’t know if i did something wrong (improbable)… Is there something like logs that i can send here so you can help me better?
The logs are in dom0 at /var/log/xen/console/guest-mirage-frewall.log (you might need to update the VM name). It should print the correct IPv4 addresses (the mirage-firewall address, its gateway address, which should be the OpenBSD address, and the DNS addresses, DNS is only used for firewall rules that involve domain names and shouldn’t be an issue for clients resolving domains).
I’m not sure what the root issue is, but you can check if your OpenBSD VM is receiving ping packets correctly with tcpdump xnf0 (or equivalent).
- If this doesn’t receive anything, you can add
-l debugto the mirage-firewall kernelopts and restart it, it should print the packet contents and routing decisions. If it does, you’ll probably have an insight of the issue with the logs, if it doesn’t, the problem is with the client AppVM. - If it works, the issue is with OpenBSD (and could be the outgoing interface configuration (e.g.
em0), the NAT/FW configuration of pf, or the IP forwarding configuration).
About the addresses, everything is ok. My OpenBSD addresses is:
IP: 10.137.0.26
Netmask: 255.255.255.255
Gateway: 10.137.0.27
Virtual DNS: 10.139.1.1, 10.139.1.2
Yes, OpenBSD is receiving ping correcly. I pretty sure that the problem its with the connection between the OpenBSD sys-net and Mirage-firewall, since Mirage-Firewall with default sys-net is working and OpenBSD is receiving internet correctly.
I tried to upload my mirage-firewall logs here but for some reason i can’t upload it. So i will send you a dump about it:
[2025-02-24 12:22:27] Logfile Opened
[2025-02-24 12:22:27] Solo5: Xen console: port 0x2, ring @0x00000000FEFFF000
[2025-02-24 12:22:27] | ___|
[2025-02-24 12:22:27] __| _ \ | _ \ __ \
[2025-02-24 12:22:27] \__ \ ( | | ( | ) |
[2025-02-24 12:22:27] ____/\___/ _|\___/____/
[2025-02-24 12:22:27] Solo5: Bindings version v0.9.0
[2025-02-24 12:22:27] Solo5: Memory map: 32 MB addressable:
[2025-02-24 12:22:27] Solo5: reserved @ (0x0 - 0xfffff)
[2025-02-24 12:22:27] Solo5: text @ (0x100000 - 0x2b7fff)
[2025-02-24 12:22:27] Solo5: rodata @ (0x2b8000 - 0x318fff)
[2025-02-24 12:22:27] Solo5: data @ (0x319000 - 0x48ffff)
[2025-02-24 12:22:27] Solo5: heap >= 0x490000 < stack < 0x2000000
[2025-02-24 12:22:27] 2025-02-24T15:22:27-00:00: [INFO] [qubes.rexec] client connected, using protocol version 3
[2025-02-24 12:22:27] 2025-02-24T15:22:27-00:00: [INFO] [qubes.db] connecting to server...
[2025-02-24 12:22:27] 2025-02-24T15:22:27-00:00: [INFO] [qubes.db] connected
[2025-02-24 12:22:27] 2025-02-24T15:22:27-00:00: [INFO] [unikernel] QubesDB and qrexec agents connected in 0.011 s
[2025-02-24 12:22:27] 2025-02-24T15:22:27-00:00: [WARNING] [dao] QubesDB key "/qubes-ip" not (yet) present; waiting for QubesDB to change...
[2025-02-24 12:22:27] 2025-02-24T15:22:27-00:00: [WARNING] [command] << Unknown command "QUBESRPC qubes.SetMonitorLayout dom0"
[2025-02-24 12:29:55] 2025-02-24T15:29:55-00:00: [INFO] [qubes.db] got update: "/connected-ips" = "10.137.0.26"
[2025-02-24 12:29:55] 2025-02-24T15:29:55-00:00: [INFO] [qubes.db] got update: "/connected-ips6" = ""
[2025-02-24 12:29:55] 2025-02-24T15:29:55-00:00: [INFO] [qubes.db] got update: "/mapped-ip/10.137.0.26/visible-ip" = "10.137.0.26"
[2025-02-24 12:29:55] 2025-02-24T15:29:55-00:00: [INFO] [qubes.db] got update: "/mapped-ip/10.137.0.26/visible-gateway" = "10.137.0.27"
[2025-02-24 12:29:55] 2025-02-24T15:29:55-00:00: [WARNING] [dao] QubesDB key "/qubes-ip" not (yet) present; waiting for QubesDB to change...
[2025-02-24 12:29:55] 2025-02-24T15:29:55-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.26/policy" = "drop"
[2025-02-24 12:29:55] 2025-02-24T15:29:55-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.26/0000" = "action=accept"
[2025-02-24 12:29:55] 2025-02-24T15:29:55-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.26" = ""
[2025-02-24 12:29:55] 2025-02-24T15:29:55-00:00: [WARNING] [dao] QubesDB key "/qubes-ip" not (yet) present; waiting for QubesDB to change...
[2025-02-24 12:37:58] Logfile Opened
[2025-02-24 12:37:58] Solo5: Xen console: port 0x2, ring @0x00000000FEFFF000
[2025-02-24 12:37:58] | ___|
[2025-02-24 12:37:58] __| _ \ | _ \ __ \
[2025-02-24 12:37:58] \__ \ ( | | ( | ) |
[2025-02-24 12:37:58] ____/\___/ _|\___/____/
[2025-02-24 12:37:58] Solo5: Bindings version v0.9.0
[2025-02-24 12:37:58] Solo5: Memory map: 32 MB addressable:
[2025-02-24 12:37:58] Solo5: reserved @ (0x0 - 0xfffff)
[2025-02-24 12:37:58] Solo5: text @ (0x100000 - 0x2b7fff)
[2025-02-24 12:37:58] Solo5: rodata @ (0x2b8000 - 0x318fff)
[2025-02-24 12:37:58] Solo5: data @ (0x319000 - 0x48ffff)
[2025-02-24 12:37:58] Solo5: heap >= 0x490000 < stack < 0x2000000
[2025-02-24 12:37:58] 2025-02-24T15:37:58-00:00: [INFO] [qubes.rexec] client connected, using protocol version 3
[2025-02-24 12:37:58] 2025-02-24T15:37:58-00:00: [INFO] [qubes.db] connecting to server...
[2025-02-24 12:37:58] 2025-02-24T15:37:58-00:00: [INFO] [qubes.db] connected
[2025-02-24 12:37:58] 2025-02-24T15:37:58-00:00: [INFO] [unikernel] QubesDB and qrexec agents connected in 0.012 s
[2025-02-24 12:37:58] 2025-02-24T15:37:58-00:00: [INFO] [dao] Current network configuration (QubesDB or command line):
[2025-02-24 12:37:58] NetVM IP on uplink network: 10.137.0.26
[2025-02-24 12:37:58] Our IP on client networks: 10.137.0.27
[2025-02-24 12:37:58] DNS primary resolver: 10.139.1.1
[2025-02-24 12:37:58] DNS secondary resolver: 10.139.1.2
[2025-02-24 12:37:58] 2025-02-24T15:37:58-00:00: [INFO] [dispatcher] Waiting for netvm changes to "/qubes-gateway"...
[2025-02-24 12:37:58] 2025-02-24T15:37:58-00:00: [INFO] [dispatcher] QubesDB has changed but not the situation of our netvm!
[2025-02-24 12:37:58] 2025-02-24T15:37:58-00:00: [INFO] [dispatcher] Waiting for netvm changes to "/qubes-gateway"...
[2025-02-24 12:37:58] 2025-02-24T15:37:58-00:00: [INFO] [dao] Watching backend/vif
[2025-02-24 12:37:58] 2025-02-24T15:37:58-00:00: [WARNING] [command] << Unknown command "QUBESRPC qubes.SetMonitorLayout dom0"
[2025-02-24 12:37:58] 2025-02-24T15:37:58-00:00: [INFO] [qubes.db] got update: "/connected-ips" = "10.137.0.26"
[2025-02-24 12:37:58] 2025-02-24T15:37:58-00:00: [INFO] [qubes.db] got update: "/connected-ips6" = ""
[2025-02-24 12:37:58] 2025-02-24T15:37:58-00:00: [INFO] [qubes.db] got update: "/mapped-ip/10.137.0.26/visible-ip" = "10.137.0.26"
[2025-02-24 12:37:58] 2025-02-24T15:37:58-00:00: [INFO] [dispatcher] QubesDB has changed but not the situation of our netvm!
[2025-02-24 12:37:58] 2025-02-24T15:37:58-00:00: [INFO] [dispatcher] Waiting for netvm changes to "/qubes-gateway"...
[2025-02-24 12:37:58] 2025-02-24T15:37:58-00:00: [INFO] [qubes.db] got update: "/mapped-ip/10.137.0.26/visible-gateway" = "10.137.0.27"
[2025-02-24 12:37:58] 2025-02-24T15:37:58-00:00: [INFO] [dispatcher] QubesDB has changed but not the situation of our netvm!
[2025-02-24 12:37:58] 2025-02-24T15:37:58-00:00: [INFO] [dispatcher] Waiting for netvm changes to "/qubes-gateway"...
[2025-02-24 12:37:58] 2025-02-24T15:37:58-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.26/policy" = "drop"
[2025-02-24 12:37:58] 2025-02-24T15:37:58-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.26/0000" = "action=accept"
[2025-02-24 12:37:58] 2025-02-24T15:37:58-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.26" = ""
[2025-02-24 12:37:58] 2025-02-24T15:37:58-00:00: [INFO] [dispatcher] QubesDB has changed but not the situation of our netvm!
[2025-02-24 12:37:58] 2025-02-24T15:37:58-00:00: [INFO] [dispatcher] Waiting for netvm changes to "/qubes-gateway"...
[2025-02-24 12:37:59] 2025-02-24T15:37:59-00:00: [INFO] [dispatcher] add client vif {domid=19;device_id=0} with IP 10.137.0.26
[2025-02-24 12:37:59] 2025-02-24T15:37:59-00:00: [INFO] [ethernet] Connected Ethernet interface fe:ff:ff:ff:ff:ff
[2025-02-24 12:37:59] 2025-02-24T15:37:59-00:00: [INFO] [dispatcher] Client 19:0 (IP: 10.137.0.26) ready
[2025-02-24 12:37:59] 2025-02-24T15:37:59-00:00: [INFO] [dispatcher] New firewall rules for 10.137.0.26
[2025-02-24 12:37:59] 0 any accept
2025-02-24T15:38:00-00:00: [INFO] [dispatcher] add client vif {domid=18;device_id=0} with IP 10.137.0.26
[2025-02-24 12:38:00] 2025-02-24T15:38:00-00:00: [INFO] [qubes.db] got rm "/qubes-firewall/10.137.0.26/"
[2025-02-24 12:38:00] 2025-02-24T15:38:00-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.26/policy" = "drop"
[2025-02-24 12:38:00] 2025-02-24T15:38:00-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.26/0000" = "action=accept"
[2025-02-24 12:38:00] 2025-02-24T15:38:00-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.26" = ""
[2025-02-24 12:38:00] 2025-02-24T15:38:00-00:00: [INFO] [dispatcher] Rules did not change for 10.137.0.26
[2025-02-24 12:38:09] 2025-02-24T15:38:09-00:00: [INFO] [ethernet] Connected Ethernet interface fe:ff:ff:ff:ff:ff
[2025-02-24 12:38:09] 2025-02-24T15:38:09-00:00: [dom18:10.137.0.26] [client_eth] Waiting for old client dom19:10.137.0.26 to go away before accepting new one
[2025-02-24 12:38:09] 2025-02-24T15:38:09-00:00: [INFO] [dispatcher] Client 18:0 (IP: 10.137.0.26) ready
[2025-02-24 12:38:09] 2025-02-24T15:38:09-00:00: [INFO] [dispatcher] New firewall rules for 10.137.0.26
[2025-02-24 12:38:09] 0 any accept
[2025-02-24 12:38:09] 2025-02-24T15:38:09-00:00: [INFO] [net-xen backend] Frontend asked to close network device dom:19/vif:0
[2025-02-24 12:38:09] 2025-02-24T15:38:09-00:00: [INFO] [dispatcher] client {domid=19;device_id=0} has gone
[2025-02-24 12:38:24] 2025-02-24T15:38:24-00:00: [dom18:10.137.0.26] [client_eth] who-has 10.137.0.26? ignoring request for client's own IP
[2025-02-24 12:39:12] 2025-02-24T15:39:12-00:00: [INFO] [qubes.db] got update: "/connected-ips" = "10.137.0.16 10.137.0.26"
[2025-02-24 12:39:12] 2025-02-24T15:39:12-00:00: [INFO] [qubes.db] got update: "/connected-ips6" = ""
[2025-02-24 12:39:12] 2025-02-24T15:39:12-00:00: [INFO] [dispatcher] QubesDB has changed but not the situation of our netvm!
[2025-02-24 12:39:12] 2025-02-24T15:39:12-00:00: [INFO] [dispatcher] Waiting for netvm changes to "/qubes-gateway"...
[2025-02-24 12:39:41] 2025-02-24T15:39:41-00:00: [INFO] [qubes.db] got update: "/connected-ips" = "10.137.0.16 10.137.0.26"
[2025-02-24 12:39:41] 2025-02-24T15:39:41-00:00: [INFO] [qubes.db] got update: "/connected-ips6" = ""
[2025-02-24 12:39:41] 2025-02-24T15:39:41-00:00: [INFO] [qubes.db] got update: "/mapped-ip/10.137.0.16/visible-ip" = "10.137.0.16"
[2025-02-24 12:39:41] 2025-02-24T15:39:41-00:00: [INFO] [dispatcher] QubesDB has changed but not the situation of our netvm!
[2025-02-24 12:39:41] 2025-02-24T15:39:41-00:00: [INFO] [dispatcher] Waiting for netvm changes to "/qubes-gateway"...
[2025-02-24 12:39:41] 2025-02-24T15:39:41-00:00: [INFO] [qubes.db] got update: "/mapped-ip/10.137.0.16/visible-gateway" = "10.137.0.27"
[2025-02-24 12:39:41] 2025-02-24T15:39:41-00:00: [INFO] [dispatcher] QubesDB has changed but not the situation of our netvm!
[2025-02-24 12:39:41] 2025-02-24T15:39:41-00:00: [INFO] [dispatcher] Waiting for netvm changes to "/qubes-gateway"...
[2025-02-24 12:39:41] 2025-02-24T15:39:41-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.16/policy" = "drop"
[2025-02-24 12:39:41] 2025-02-24T15:39:41-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.16/0000" = "action=accept"
[2025-02-24 12:39:41] 2025-02-24T15:39:41-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.16" = ""
[2025-02-24 12:39:41] 2025-02-24T15:39:41-00:00: [INFO] [dispatcher] QubesDB has changed but not the situation of our netvm!
[2025-02-24 12:39:41] 2025-02-24T15:39:41-00:00: [INFO] [dispatcher] Waiting for netvm changes to "/qubes-gateway"...
[2025-02-24 12:39:42] 2025-02-24T15:39:42-00:00: [INFO] [dispatcher] add client vif {domid=20;device_id=0} with IP 10.137.0.16
[2025-02-24 12:39:44] 2025-02-24T15:39:44-00:00: [INFO] [ethernet] Connected Ethernet interface fe:ff:ff:ff:ff:ff
[2025-02-24 12:39:44] 2025-02-24T15:39:44-00:00: [INFO] [dispatcher] Client 20:0 (IP: 10.137.0.16) ready
[2025-02-24 12:39:44] 2025-02-24T15:39:44-00:00: [INFO] [dispatcher] New firewall rules for 10.137.0.16
[2025-02-24 12:39:44] 0 any accept
[2025-02-24 12:39:45] 2025-02-24T15:39:45-00:00: [INFO] [qubes.db] got rm "/qubes-firewall/10.137.0.16/"
[2025-02-24 12:39:45] 2025-02-24T15:39:45-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.16/policy" = "drop"
[2025-02-24 12:39:45] 2025-02-24T15:39:45-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.16/0000" = "action=accept"
[2025-02-24 12:39:45] 2025-02-24T15:39:45-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.16" = ""
[2025-02-24 12:39:45] 2025-02-24T15:39:45-00:00: [INFO] [dispatcher] Rules did not change for 10.137.0.16
2025-02-24T15:52:55-00:00: [WARNING] [net-xen xenstore] Error reading device state at "/local/domain/18/device/vif/0": Xs_protocol.Error("EACCES")
[2025-02-24 12:52:55] 2025-02-24T15:52:55-00:00: [INFO] [net-xen backend] Frontend asked to close network device dom:18/vif:0
[2025-02-24 12:52:55] 2025-02-24T15:52:55-00:00: [WARNING] [net-xen xenstore] Error reading device state at "/local/domain/18/device/vif/0": Xs_protocol.Error("EACCES")
[2025-02-24 12:52:55] 2025-02-24T15:52:55-00:00: [INFO] [dispatcher] client {domid=18;device_id=0} has gone
[2025-02-24 12:52:55] 2025-02-24T15:52:55-00:00: [ERROR] [dispatcher] We have a command line configuration 10.137.0.26 but it's currently not connected to us (please check its netvm property)...
[2025-02-24 12:52:56] 2025-02-24T15:52:56-00:00: [ERROR] [dispatcher] We have a command line configuration 10.137.0.26 but it's currently not connected to us (please check its netvm property)...
[2025-02-24 12:52:57] 2025-02-24T15:52:57-00:00: [ERROR] [dispatcher] We have a command line configuration 10.137.0.26 but it's currently not connected to us (please check its netvm property)...
[2025-02-24 12:52:58] 2025-02-24T15:52:58-00:00: [ERROR] [dispatcher] We have a command line configuration 10.137.0.26 but it's currently not connected to us (please check its netvm property)...
[2025-02-24 12:52:58] 2025-02-24T15:52:58-00:00: [ERROR] [dispatcher] We have a command line configuration 10.137.0.26 but it's currently not connected to us (please check its netvm property)...
[2025-02-24 12:52:59] 2025-02-24T15:52:59-00:00: [ERROR] [dispatcher] We have a command line configuration 10.137.0.26 but it's currently not connected to us (please check its netvm property)...
[2025-02-24 12:53:00] 2025-02-24T15:53:00-00:00: [ERROR] [dispatcher] We have a command line configuration 10.137.0.26 but it's currently not connected to us (please check its netvm property)...
[2025-02-24 12:53:01] 2025-02-24T15:53:01-00:00: [ERROR] [dispatcher] We have a command line configuration 10.137.0.26 but it's currently not connected to us (please check its netvm property)...
[2025-02-24 12:53:02] 2025-02-24T15:53:02-00:00: [ERROR] [dispatcher] We have a command line configuration 10.137.0.26 but it's currently not connected to us (please check its netvm property)...
[2025-02-24 12:53:03] 2025-02-24T15:53:03-00:00: [ERROR] [dispatcher] We have a command line configuration 10.137.0.26 but it's currently not connected to us (please check its netvm property)...
[2025-02-24 12:53:04] 2025-02-24T15:53:04-00:00: [ERROR] [dispatcher] We have a command line configuration 10.137.0.26 but it's currently not connected to us (please check its netvm property)...
[2025-02-24 12:53:05] 2025-02-24T15:53:05-00:00: [ERROR] [dispatcher] We have a command line configuration 10.137.0.26 but it's currently not connected to us (please check its netvm property)...
[2025-02-24 12:53:07] 2025-02-24T15:53:07-00:00: [ERROR] [dispatcher] We have a command line configuration 10.137.0.26 but it's currently not connected to us (please check its netvm property)...
[2025-02-24 12:53:08] 2025-02-24T15:53:08-00:00: [ERROR] [dispatcher] We have a command line configuration 10.137.0.26 but it's currently not connected to us (please check its netvm property)...
[2025-02-24 12:53:09] 2025-02-24T15:53:09-00:00: [ERROR] [dispatcher] We have a command line configuration 10.137.0.26 but it's currently not connected to us (please check its netvm property)...
[2025-02-24 12:53:10] 2025-02-24T15:53:10-00:00: [ERROR] [dispatcher] We have a command line configuration 10.137.0.26 but it's currently not connected to us (please check its netvm property)...
2025-02-24T15:53:11-00:00: [ERROR] [dispatcher] We have a command line configuration 10.137.0.26 but it's currently not connected to us (please check its netvm property)...
[2025-02-24 12:53:12] 2025-02-24T15:53:12-00:00: [ERROR] [dispatcher] We have a command line configuration 10.137.0.26 but it's currently not connected to us (please check its netvm property)...
2025-02-24T15:53:13-00:00: [ERROR] [dispatcher] We have a command line configuration 10.137.0.26 but it's currently not connected to us (please check its netvm property)...
[2025-02-24 12:53:14] 2025-02-24T15:53:14-00:00: [ERROR] [dispatcher] We have a command line configuration 10.137.0.26 but it's currently not connected to us (please check its netvm property)...
[2025-02-24 12:53:14] 2025-02-24T15:53:14-00:00: [ERROR] [dispatcher] We have a command line configuration 10.137.0.26 but it's currently not connected to us (please check its netvm property)...
[2025-02-24 12:53:15] 2025-02-24T15:53:15-00:00: [ERROR] [dispatcher] We have a command line configuration 10.137.0.26 but it's currently not connected to us (please check its netvm property)...
[2025-02-24 12:53:16] 2025-02-24T15:53:16-00:00: [ERROR] [dispatcher] We have a command line configuration 10.137.0.26 but it's currently not connected to us (please check its netvm property)...
[2025-02-24 12:53:17] 2025-02-24T15:53:17-00:00: [ERROR] [dispatcher] We have a command line configuration 10.137.0.26 but it's currently not connected to us (please check its netvm property)...
[2025-02-24 12:53:18] 2025-02-24T15:53:18-00:00: [ERROR] [dispatcher] We have a command line configuration 10.137.0.26 but it's currently not connected to us (please check its netvm property)...
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [ERROR] [dispatcher] We have a command line configuration 10.137.0.26 but it's currently not connected to us (please check its netvm property)...
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/name" = "mirage-firewall"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/type" = "StandaloneVM"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/default-user" = "user"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/qubes-vm-updateable" = "True"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/qubes-vm-persistence" = "full"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/qubes-debug-mode" = "0"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/qubes-base-template" = ""
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/qubes-random-seed" = "Wn9/ky/odhXwSovLFK+ba2ZazPYnmWqcH+cGEvPaTi6DPxs72VAaEreLH1KOb6u+VAwlND11PS3QIkhcHtI5mw=="
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [dispatcher] QubesDB has changed but not the situation of our netvm!
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [dispatcher] Waiting for netvm changes to "/qubes-gateway"...
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/qubes-netvm-network" = "10.137.0.27"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/qubes-netvm-gateway" = "10.137.0.27"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/qubes-netvm-netmask" = "255.255.255.255"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/qubes-netvm-primary-dns" = "10.139.1.1"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/qubes-netvm-secondary-dns" = "10.139.1.2"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/qubes-mac" = "00:16:3e:5e:6c:00"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [dispatcher] QubesDB has changed but not the situation of our netvm!
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [dispatcher] Waiting for netvm changes to "/qubes-gateway"...
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/qubes-ip" = "10.137.0.27"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/qubes-netmask" = "255.255.255.255"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/qubes-gateway" = "10.137.0.6"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [dispatcher] Going from netvm not connected to /qubes-gateway
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [WARNING] [dao] QubesDB key "/qubes-primary-dns" not (yet) present; waiting for QubesDB to change...
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/qubes-primary-dns" = "10.139.1.1"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/qubes-secondary-dns" = "10.139.1.2"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [dao] Current network configuration (QubesDB or command line):
[2025-02-24 12:53:19] NetVM IP on uplink network: 10.137.0.6
[2025-02-24 12:53:19] Our IP on client networks: 10.137.0.27
[2025-02-24 12:53:19] DNS primary resolver: 10.139.1.1
[2025-02-24 12:53:19] DNS secondary resolver: 10.139.1.2
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [net-xen frontend] connect 0
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/qubes-timezone" = "America/Sao_Paulo"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/qubes-block-devices" = ""
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/qubes-usb-devices" = ""
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/qubes-audio-domain-xid" = "0"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/qubes-vm-type" = "ProxyVM"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/qubes-iptables-error" = ""
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/keyboard-layout" = "br+thinkpad+"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/qubes-gui-domain-xid" = "0"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/qubes-service/meminfo-writer" = "1"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/connected-ips" = "10.137.0.16 10.137.0.26"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/connected-ips6" = ""
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/mapped-ip/10.137.0.16/visible-ip" = "10.137.0.16"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/mapped-ip/10.137.0.16/visible-gateway" = "10.137.0.27"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got rm "/qubes-firewall/10.137.0.16/"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.16/policy" = "drop"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.16/0000" = "action=accept"
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.16" = ""
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [net-xen frontend] create: id=0 domid=21
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [net-xen frontend] sg:true gso_tcpv4:true rx_copy:true rx_flip:false smart_poll:false
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [net-xen frontend] MAC: 00:16:3e:5e:6c:00
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [ethernet] Connected Ethernet interface 00:16:3e:5e:6c:00
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [ARP] Sending gratuitous ARP for 10.137.0.27 (00:16:3e:5e:6c:00)
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [ARP] Sending gratuitous ARP for 10.137.0.27 (00:16:3e:5e:6c:00)
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [udp] UDP layer connected on 10.137.0.27/0
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [dispatcher] Waiting for netvm changes to "/qubes-gateway"...
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [dispatcher] QubesDB has changed but not the situation of our netvm!
[2025-02-24 12:53:19] 2025-02-24T15:53:19-00:00: [INFO] [dispatcher] Waiting for netvm changes to "/qubes-gateway"...
2025-02-24T16:04:21-00:00: [INFO] [net-xen backend] Frontend asked to close network device dom:20/vif:0
[2025-02-24 13:04:21] 2025-02-24T16:04:21-00:00: [INFO] [dispatcher] client {domid=20;device_id=0} has gone
[2025-02-24 13:04:21] 2025-02-24T16:04:21-00:00: [INFO] [qubes.db] got update: "/connected-ips" = "10.137.0.26"
[2025-02-24 13:04:21] 2025-02-24T16:04:21-00:00: [INFO] [qubes.db] got update: "/connected-ips6" = ""
[2025-02-24 13:04:21] 2025-02-24T16:04:21-00:00: [INFO] [dispatcher] QubesDB has changed but not the situation of our netvm!
[2025-02-24 13:04:21] 2025-02-24T16:04:21-00:00: [INFO] [dispatcher] Waiting for netvm changes to "/qubes-gateway"...
[2025-02-24 13:04:52] 2025-02-24T16:04:52-00:00: [INFO] [qubes.db] got update: "/connected-ips" = "10.137.0.26"
[2025-02-24 13:04:52] 2025-02-24T16:04:52-00:00: [INFO] [qubes.db] got update: "/connected-ips6" = ""
[2025-02-24 13:04:52] 2025-02-24T16:04:52-00:00: [INFO] [qubes.db] got update: "/mapped-ip/10.137.0.26/visible-ip" = "10.137.0.26"
[2025-02-24 13:04:52] 2025-02-24T16:04:52-00:00: [INFO] [qubes.db] got update: "/mapped-ip/10.137.0.26/visible-gateway" = "10.137.0.27"
[2025-02-24 13:04:52] 2025-02-24T16:04:52-00:00: [INFO] [qubes.db] got rm "/qubes-firewall/10.137.0.26/"
[2025-02-24 13:04:52] 2025-02-24T16:04:52-00:00: [INFO] [dispatcher] QubesDB has changed but not the situation of our netvm!
[2025-02-24 13:04:52] 2025-02-24T16:04:52-00:00: [INFO] [dispatcher] Waiting for netvm changes to "/qubes-gateway"...
[2025-02-24 13:04:52] 2025-02-24T16:04:52-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.26/policy" = "drop"
[2025-02-24 13:04:52] 2025-02-24T16:04:52-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.26/0000" = "action=accept"
[2025-02-24 13:04:52] 2025-02-24T16:04:52-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.26" = ""
[2025-02-24 13:04:52] 2025-02-24T16:04:52-00:00: [INFO] [dispatcher] QubesDB has changed but not the situation of our netvm!
[2025-02-24 13:04:52] 2025-02-24T16:04:52-00:00: [INFO] [dispatcher] Waiting for netvm changes to "/qubes-gateway"...
[2025-02-24 13:04:52] 2025-02-24T16:04:52-00:00: [INFO] [dispatcher] add client vif {domid=24;device_id=0} with IP 10.137.0.26
[2025-02-24 13:04:53] 2025-02-24T16:04:53-00:00: [INFO] [ethernet] Connected Ethernet interface fe:ff:ff:ff:ff:ff
[2025-02-24 13:04:53] 2025-02-24T16:04:53-00:00: [INFO] [dispatcher] Client 24:0 (IP: 10.137.0.26) ready
[2025-02-24 13:04:53] 2025-02-24T16:04:53-00:00: [INFO] [dispatcher] New firewall rules for 10.137.0.26
[2025-02-24 13:04:53] 0 any accept
2025-02-24T16:04:53-00:00: [INFO] [dispatcher] add client vif {domid=23;device_id=0} with IP 10.137.0.26
[2025-02-24 13:04:54] 2025-02-24T16:04:54-00:00: [INFO] [qubes.db] got rm "/qubes-firewall/10.137.0.26/"
[2025-02-24 13:04:54] 2025-02-24T16:04:54-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.26/policy" = "drop"
[2025-02-24 13:04:54] 2025-02-24T16:04:54-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.26/0000" = "action=accept"
[2025-02-24 13:04:54] 2025-02-24T16:04:54-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.26" = ""
[2025-02-24 13:04:54] 2025-02-24T16:04:54-00:00: [INFO] [dispatcher] Rules did not change for 10.137.0.26
[2025-02-24 13:05:02] 2025-02-24T16:05:02-00:00: [INFO] [ethernet] Connected Ethernet interface fe:ff:ff:ff:ff:ff
[2025-02-24 13:05:02] 2025-02-24T16:05:02-00:00: [dom23:10.137.0.26] [client_eth] Waiting for old client dom24:10.137.0.26 to go away before accepting new one
[2025-02-24 13:05:02] 2025-02-24T16:05:02-00:00: [INFO] [dispatcher] Client 23:0 (IP: 10.137.0.26) ready
[2025-02-24 13:05:02] 2025-02-24T16:05:02-00:00: [INFO] [dispatcher] New firewall rules for 10.137.0.26
[2025-02-24 13:05:02] 0 any accept
[2025-02-24 13:05:02] 2025-02-24T16:05:02-00:00: [INFO] [net-xen backend] Frontend asked to close network device dom:24/vif:0
[2025-02-24 13:05:02] 2025-02-24T16:05:02-00:00: [INFO] [dispatcher] client {domid=24;device_id=0} has gone
[2025-02-24 13:05:17] 2025-02-24T16:05:17-00:00: [dom23:10.137.0.26] [client_eth] who-has 10.137.0.26? ignoring request for client's own IP
[2025-02-24 13:07:19] 2025-02-24T16:07:19-00:00: [INFO] [qubes.db] got update: "/connected-ips" = "10.137.0.16 10.137.0.26"
[2025-02-24 13:07:19] 2025-02-24T16:07:19-00:00: [INFO] [qubes.db] got update: "/connected-ips6" = ""
[2025-02-24 13:07:19] 2025-02-24T16:07:19-00:00: [INFO] [dispatcher] QubesDB has changed but not the situation of our netvm!
[2025-02-24 13:07:19] 2025-02-24T16:07:19-00:00: [INFO] [dispatcher] Waiting for netvm changes to "/qubes-gateway"...
[2025-02-24 13:07:25] 2025-02-24T16:07:25-00:00: [INFO] [qubes.db] got update: "/connected-ips" = "10.137.0.16 10.137.0.26"
[2025-02-24 13:07:25] 2025-02-24T16:07:25-00:00: [INFO] [qubes.db] got update: "/connected-ips6" = ""
[2025-02-24 13:07:25] 2025-02-24T16:07:25-00:00: [INFO] [qubes.db] got update: "/mapped-ip/10.137.0.16/visible-ip" = "10.137.0.16"
[2025-02-24 13:07:25] 2025-02-24T16:07:25-00:00: [INFO] [qubes.db] got update: "/mapped-ip/10.137.0.16/visible-gateway" = "10.137.0.27"
[2025-02-24 13:07:25] 2025-02-24T16:07:25-00:00: [INFO] [qubes.db] got rm "/qubes-firewall/10.137.0.16/"
[2025-02-24 13:07:25] 2025-02-24T16:07:25-00:00: [INFO] [dispatcher] QubesDB has changed but not the situation of our netvm!
[2025-02-24 13:07:25] 2025-02-24T16:07:25-00:00: [INFO] [dispatcher] Waiting for netvm changes to "/qubes-gateway"...
[2025-02-24 13:07:25] 2025-02-24T16:07:25-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.16/policy" = "drop"
[2025-02-24 13:07:25] 2025-02-24T16:07:25-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.16/0000" = "action=accept"
[2025-02-24 13:07:25] 2025-02-24T16:07:25-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.16" = ""
[2025-02-24 13:07:25] 2025-02-24T16:07:25-00:00: [INFO] [dispatcher] QubesDB has changed but not the situation of our netvm!
[2025-02-24 13:07:25] 2025-02-24T16:07:25-00:00: [INFO] [dispatcher] Waiting for netvm changes to "/qubes-gateway"...
[2025-02-24 13:07:26] 2025-02-24T16:07:26-00:00: [INFO] [dispatcher] add client vif {domid=25;device_id=0} with IP 10.137.0.16
[2025-02-24 13:07:29] 2025-02-24T16:07:29-00:00: [INFO] [ethernet] Connected Ethernet interface fe:ff:ff:ff:ff:ff
[2025-02-24 13:07:29] 2025-02-24T16:07:29-00:00: [INFO] [dispatcher] Client 25:0 (IP: 10.137.0.16) ready
[2025-02-24 13:07:29] 2025-02-24T16:07:29-00:00: [INFO] [dispatcher] New firewall rules for 10.137.0.16
[2025-02-24 13:07:29] 0 any accept
[2025-02-24 13:07:30] 2025-02-24T16:07:30-00:00: [INFO] [qubes.db] got rm "/qubes-firewall/10.137.0.16/"
[2025-02-24 13:07:30] 2025-02-24T16:07:30-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.16/policy" = "drop"
[2025-02-24 13:07:30] 2025-02-24T16:07:30-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.16/0000" = "action=accept"
[2025-02-24 13:07:30] 2025-02-24T16:07:30-00:00: [INFO] [qubes.db] got update: "/qubes-firewall/10.137.0.16" = ""
[2025-02-24 13:07:30] 2025-02-24T16:07:30-00:00: [INFO] [dispatcher] Rules did not change for 10.137.0.16
2025-02-24T16:20:12-00:00: [WARNING] [net-xen xenstore] Error reading device state at "/local/domain/23/device/vif/0": Xs_protocol.Error("EACCES")
[2025-02-24 13:20:12] 2025-02-24T16:20:12-00:00: [INFO] [net-xen backend] Frontend asked to close network device dom:23/vif:0
[2025-02-24 13:20:12] 2025-02-24T16:20:12-00:00: [WARNING] [net-xen xenstore] Error reading device state at "/local/domain/23/device/vif/0": Xs_protocol.Error("EACCES")
[2025-02-24 13:20:12] 2025-02-24T16:20:12-00:00: [INFO] [dispatcher] client {domid=23;device_id=0} has gone
[2025-02-24 13:20:25] 2025-02-24T16:20:25-00:00: [INFO] [net-xen backend] Frontend asked to close network device dom:25/vif:0
[2025-02-24 13:20:25] 2025-02-24T16:20:25-00:00: [INFO] [dispatcher] client {domid=25;device_id=0} has gone
[2025-02-24 13:20:35] Solo5: solo5_exit(0) called
Can you try to print your routing table? The default gateway of your OpenBSD VM should be the one provided by DHCP/any mechanism on your “not virtual” interface, otherwise the packet can’t get out of your laptop.
2 Likes
That line could also be an issue, can you double check mirage-firewall has no netvm configured? I mean mirage-firewall has find the kernelopts configuration (the NetVM IP printed at startup is correct), but a bit later it changes to 10.137.0.6, maybe that is because you changed the netvm property when it’s running? In the case of this guide setup, mirage-firewall should have no netvm, in more classic setup it has sys-net as netvm.
1 Like
This is probably because i changed the NetVM of mirage-firewall to “sys-net” (The default of Qubes OS) to test if the problem is from OpenBSD or from mirage-firewall.
I will send you some screenshots about logs and tcpdump. A lot of strange things happened here.
These first 2 images are the output of Routing Tables. The third image is the ping 8.8.8.8, so you can see that the internet is working perfectly in OpenBSD Vm.
But for now, the things will be so strange.
This is the tcpdump xnf0 output. Note that in background im trying to access youtube with a a qube named “personal”, with NetVM mirage-firewall. Tcpdump succesfully identifies that im trying to access the internet, but it just doens’t access in the end. Just a loading loop of the site.
Re0 tcpdump (My Attached network interface)
It seems from the 4th capture that openBSD doesn’t forward back the DNS replies. When hunting similar issue with @qEawma5f , the root cause was the DNS rules in pf.conf if I recall correctly so I don’t really understand why this fails as the modification is only 4 lines 
2 Likes
Reinstall and make sure the DNS nameserver of xnf0 is the ip of your virtual dns reported inside Qubes Manager
1 Like
Thanks for all support guys! When i get back to home, i will test changing my DNS nameservers to make sure if i don’t misconfigure something, and i will come back to tell you what happened.
Updating here: I tried everything but it still doens’t works.
This is my /etc/resolv.conf. I saw that it was missing the dns reported inside Qubes manager, since the 1st dns is 10.139.1.1 and the 2nd 10.139.1.2 . So i delete everything and left just that two DNS with nameserver and tried again, and the tcpdump -i xnf0 just doens’t give any logs with that config. So i decided to try to maintain the “192.168…” nameserver and just add the 1st dns from qube manager that was missing,
With that, the log output back to work but, again, the same error. No internet connection. Trying to ping from the personal qube and it just gave me these logs:
I don’t know what more i can try… Any idea?
There should be no 10.139.* in /etc/resolv.conf: these addresses are only internal to Qubes and will be resolved by the net VM according to its configuration (e.g. your 192.168.18.1). This is the purpose of match in proto { udp tcp } from xnf0:network ... and pass in proto... in the pf.conf guide.
Your resolv.conf will be correct with only the first line (or automatically generated by dhclient?), and it seems to resolve correctly from the tcpdump traces in your previous post.
In the last image, 10.137.0.27 is sending ICMP requests, but there is no response. As it seems, from the 5th image in your previous post, there is also traffic on re0, so IP forwarding is correctly enabled. It also appears that the NAT process is correct as the outgoing packets have the IP address of re0 (192.168.18.109) as their source address. So the packets seem to be lost on the way back, either because they are not NATed back or because they are filtered. I don’t know enough about OpenBSD to track deeper on this issue
1 Like
Ah, i understand it now. I thought that i need to add these dns addresses in /etc/resolv.conf, since @qEawma5f said that dns nameserver of xnf0 should be the virtual dns reported inside qube manager… During the installation of OpenBSD, the installer didn’t ask me about gateway’s ip and the dns nameserver.
So i had to set it up post-installation. Probably i made some misconfiguration in these steps, idk… But for now, thanks you all for the help. Soon i will try to solve this problem. I will make a deeper look about the dns addresses.
Usually, manual modifications to the /etc/resolv.conf (e.g adding the dns by yourself) file will be overwritten by DHCP. However, you can configure the DHCP to ignore these changes regarding DNS settings. I don’t remember the exact steps, but it is certainly possible.
I can’t see all the details in the images, so maybe you tried this, but can your qube successfully ping its upstream gateway? I think ping is not normally blocked.
(I was wondering about the netmask on the interface… although maybe that is normal)
What would it take to port guest tools to OpenBSD, qrexec and update at least?
The setting you are thinking of is in /etc/dhcpleased.conf:
interface em0 ignore dns
This will let dhcp run on em0, but will not update /etc/resolv.conf.
You certainly dont want to use 10.139* addresses in /etc/resolv.conf
so if this is in the guide it seems like a mistake. (Unless the external
network is a 10. network, in which case you would do better to change
the internal addresses used by Qubes to avoid issues.)
OP, have you resolved your issues yet?
I never presume to speak for the Qubes team.
When I comment in the Forum I speak for myself.
1 Like