Ethernet not working

User Support General
1

I have a little bit of a problem getting my internet connection to work…

Quick overview:

  • Internal WiFi card & USB WiFi are ruled out, internal card causes the system to crash, external doesn’t work because (apparently) I’ve only got 1 USB controller → needs to be at dom0
  • Drivers for ethernet device (apparently) are available at sys-net (I guess it’s the best option for me to have the internet connection there?)
  • lspci in dom0 shows the card, as well as in sys-net
  • I attached the device to sys-net using:
    qvm-pci attach --persistent --option permissive=true --option no-strict-reset=true sys-usb dom0:<BDF_OF_DEVICE>
  • lspci -v in sys-net gives following output, regarding the ethernet device:
00:06.0 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM57766 Gigabit Ethernet PCIe (rev 01)
	Subsystem: Broadcom Inc. and subsidiaries NetXtreme BCM57766 Gigabit Ethernet PCIe
	Physical Slot: 6
	Flags: fast devsel, IRQ 40
	Memory at f2010000 (64-bit, prefetchable) [size=64K]
	Memory at f2020000 (64-bit, prefetchable) [size=64K]
	Capabilities: <access denied>
	Kernel modules: tg3
  • Drivers should be installed (t3), as it is in the firmware-misc-nonfree package and it was already installed in the debian-11 template when I tried to install it.

Conclusion:
The network manager (icon on the top right) shows “No network devices available”, just the same as before I attached the ethernet device to sys-net… Could there be anything I’ve missed? It could only be a problem with the drivers, couldn’t it? But they should be installed already… It works on standalone debian, just can’t get it to work in Qubes.
If maybe somebody noticed that I missed something obvious or maybe has a little hint for a direction in which I could look further for troubleshooting, I would of course be more than thankful!

2

please post back lspci with sudo, for the beginning.

3

I did it again with sudo as you suggested, here are some outputs:

sudo lspci in dom0 
00:00.0 Host bridge: Intel Corporation Xeon E3-1200 v6/7th Gen Core Processor Host Bridge/DRAM Registers (rev 05)
00:01.0 PCI bridge: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) (rev 05)
00:14.0 USB controller: Intel Corporation 100 Series/C230 Series Chipset Family USB 3.0 xHCI Controller (rev 31)
00:16.0 Communication controller: Intel Corporation 100 Series/C230 Series Chipset Family MEI Controller #1 (rev 31)
00:17.0 SATA controller: Intel Corporation Q170/Q150/B150/H170/H110/Z170/CM236 Chipset SATA Controller [AHCI Mode] (rev 31)
00:1b.0 PCI bridge: Intel Corporation 100 Series/C230 Series Chipset Family PCI Express Root Port #17 (rev f1)
00:1c.0 PCI bridge: Intel Corporation 100 Series/C230 Series Chipset Family PCI Express Root Port #1 (rev f1)
00:1c.1 PCI bridge: Intel Corporation 100 Series/C230 Series Chipset Family PCI Express Root Port #2 (rev f1)
00:1c.4 PCI bridge: Intel Corporation 100 Series/C230 Series Chipset Family PCI Express Root Port #5 (rev f1)
00:1f.0 ISA bridge: Intel Corporation Z170 Chipset LPC/eSPI Controller (rev 31)
00:1f.2 Memory controller: Intel Corporation 100 Series/C230 Series Chipset Family Power Management Controller (rev 31)
00:1f.3 Audio device: Intel Corporation 100 Series/C230 Series Chipset Family HD Audio Controller (rev 31)
00:1f.4 SMBus: Intel Corporation 100 Series/C230 Series Chipset Family SMBus (rev 31)
01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] (rev c5)
01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590]
02:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd Device a806
03:00.0 Network controller: Broadcom Inc. and subsidiaries BCM43602 802.11ac Wireless LAN SoC (rev 01)
04:00.0 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM57766 Gigabit Ethernet PCIe (rev 01)
04:00.1 SD Host controller: Broadcom Inc. and subsidiaries BCM57765/57785 SDXC/MMC Card Reader (rev 01)
05:00.0 PCI bridge: Intel Corporation DSL6540 Thunderbolt 3 Bridge [Alpine Ridge 4C 2015] (rev 02)
06:00.0 PCI bridge: Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] (rev 02)
06:01.0 PCI bridge: Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] (rev 02)
06:02.0 PCI bridge: Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] (rev 02)
06:04.0 PCI bridge: Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] (rev 02)
07:00.0 System peripheral: Intel Corporation JHL6540 Thunderbolt 3 NHI (C step) [Alpine Ridge 4C 2016] (rev 02)
08:00.0 USB controller: Intel Corporation JHL6540 Thunderbolt 3 USB Controller (C step) [Alpine Ridge 4C 2016] (rev 02)
sudo lspci -v in dom0 (for better overview I only post the ethernet device) 
04:00.0 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM57766 Gigabit Ethernet PCIe (rev 01)
	Subsystem: Broadcom Inc. and subsidiaries NetXtreme BCM57766 Gigabit Ethernet PCIe
	Flags: fast devsel, IRQ 17
	Memory at d0200000 (64-bit, prefetchable) [size=64K]
	Memory at d0210000 (64-bit, prefetchable) [size=64K]
	Capabilities: [48] Power Management version 3
	Capabilities: [50] Vital Product Data
	Capabilities: [58] MSI: Enable- Count=1/8 Maskable- 64bit+
	Capabilities: [a0] MSI-X: Enable- Count=6 Masked-
	Capabilities: [ac] Express Endpoint, MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [13c] Device Serial Number 00-00-78-7b-8a-be-01-09
	Capabilities: [150] Power Budgeting <?>
	Capabilities: [160] Virtual Channel
	Capabilities: [1b0] Latency Tolerance Reporting
	Kernel driver in use: pciback
	Kernel modules: tg3
sudo lspci in sys-net 
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01)
00:04.0 VGA compatible controller: Device 1234:1111 (rev 02)
00:05.0 USB controller: Intel Corporation 82801DB/DBM (ICH4/ICH4-M) USB2 EHCI Controller (rev 10)
00:06.0 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM57766 Gigabit Ethernet PCIe (rev 01)
sudo lspci -v in sys-net 
00:06.0 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM57766 Gigabit Ethernet PCIe (rev 01)
	Subsystem: Broadcom Inc. and subsidiaries NetXtreme BCM57766 Gigabit Ethernet PCIe
	Physical Slot: 6
	Flags: fast devsel, IRQ 40
	Memory at f2010000 (64-bit, prefetchable) [size=64K]
	Memory at f2020000 (64-bit, prefetchable) [size=64K]
	Capabilities: [48] Power Management version 3
	Capabilities: [50] Vital Product Data
	Capabilities: [58] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [ac] Express Endpoint, MSI 00
	Kernel modules: tg3
sudo journalctl | grep 'tg3' in sys-net 
Dec 25 08:33:34 sys-net kernel: tg3 0000:00:06.0: tg3_test_dma: Buffer write failed. err = -19
Dec 25 08:33:34 sys-net kernel: tg3 0000:00:06.0: DMA engine test failed, aborting

I also tried to attach the ethernet PCI device with sudo xl pci-attach sys-net '04:00.0,permissive=1 in dom0 instead of qvm-pci attach --persistent --option permissive=true --option no-strict-reset=true sys-net dom0:04_00.0 but this is the output when I do it:

libxl: error: libxl_pci.c:1608:libxl__device_pci_add: Domain 14:PCI device 0000:04:00.0 cannot be assigned - no IOMMU?
libxl: error: libxl_pci.c:1774:device_pci_add_done: Domain 14:libxl__device_pci_add  failed for PCI device 0:4:0.0 (rc -1)
libxl: error: libxl_device.c:1450:device_addrm_aocomplete: unable to add device

when attaching with the previous method, the attaching apparently works, as the logs I posted above with sudo.

Also, I tried to switch sys-net from HVM to PV, but this doesn’t work at all. When I switch it to PV mode (while ethernet device attached) and try to start sys-net, I get following error:

Start failed: internal error: Unable to reset PCI device 0000:04:00.0: internal error: Active 0000:04:00.1 devices on bus with 0000:04:00.0 not doing bus reset, see /var/log/libvirt/libxl/libxl-driver.log
Content of the log from the PV mode failure: 
2022-12-24 13:19:07.963+0000: libxl: libxl_pci.c:1484:libxl__device_pci_reset: write to /sys/bus/pci/devices/0000:04:00.0/reset returned -1: Inappropriate ioctl for device
2022-12-24 13:21:23.626+0000: libxl: libxl_pci.c:1484:libxl__device_pci_reset: write to /sys/bus/pci/devices/0000:04:00.0/reset returned -1: Inappropriate ioctl for device
2022-12-24 13:21:24.519+0000: libxl: libxl_pci.c:1484:libxl__device_pci_reset: write to /sys/bus/pci/devices/0000:04:00.0/reset returned -1: Inappropriate ioctl for device
2022-12-24 13:21:34.619+0000: libxl: libxl_device.c:1434:libxl__wait_for_backend: Backend /local/domain/0/backend/pci/28/0 not ready
2022-12-24 13:21:51.866+0000: libxl: libxl_pci.c:1484:libxl__device_pci_reset: write to /sys/bus/pci/devices/0000:04:00.0/reset returned -1: Inappropriate ioctl for device
2022-12-24 13:21:52.760+0000: libxl: libxl_pci.c:1484:libxl__device_pci_reset: write to /sys/bus/pci/devices/0000:04:00.0/reset returned -1: Inappropriate ioctl for device
2022-12-24 13:22:02.870+0000: libxl: libxl_device.c:1434:libxl__wait_for_backend: Backend /local/domain/0/backend/pci/29/0 not ready

But PV mode isn’t needed I guess, because in HVM I can attach the device with the command I mentioned earlier, so I think this particular error can be neglected? Main thing is to find out why the network manager only gives me No networking devices available as option with an apparently working and attached ethernet device :-/

4

First, you can try this

Please note the syntax that worked for the user

qvm-pci attach sys-net dom0:02_00.0 --persistent -o permissive=true

For example, this one worked for me:

qvm-pci attach --persistent sys-net dom0:00_1d.0 -o no-strict-reset=true

Next, and probably not the last, what worked for some of my cards was to start them as systemd services.

Just create /etc/systemd/system/04-ethernet-permissive.service file and enable service, reboot and check

[Unit]
Description=Set permissive mode to ethernet card and enable with `sudo systemctl enable 04-ethernet-permissive.service`

[Service]
ExecStart=/bin/sh -c 'echo 0000:04:00.0 > /sys/bus/pci/drivers/pciback/permissive'
Type=oneshot
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

Try it in PV mode as well as in fedora-36.

5

Alright, thank you so much so far!

I already followed this guide/thread when I was still trying to get my WiFi to work, I came back to it to try again when I gave up the idea of WiFi and tried to just get ethernet to work. But apparently the user had to put the VM in PV mode to get ethernet to work, but this last step I couldn’t do, because I can’t start Qubes in PV mode

Network card on Mac mini 7,1

I tried it with detached device too.

Thanks for this suggestion, hadn’t tried that so far! Unfortunately, the issue persists. I actually had to create the 04-ethernet-permissive.service in /lib/systemd/system/ instead of /etc/systemd/system because when I unmasked it, it got deleted. But after enabling the service in the /lib/ path it got created in /etc/ too.

Still No networking devices available in sys-net

Tried it with fedora as default of everything, too. PV mode is the only thing I can’t try…

6

I am sorry I cannot be of a greater help here, but for what is worth, there is enough info now for others with different ideas that will hopefully show up here.
Maybe you should give additional info about kernel, Xen and other modules versions, to try with kernel-latest if not, to try with the different kernels, and so on…

1 Like