System Details
- Lenovo ThinkPad P14s Gen4 AMD
- Qualcomm QCNFA765 (wcn6855 hw2.1)
- Qubes OS 4.3.0
Problem
The ath11k driver fails with PCI passthrough. The driver reads MSI configuration directly from hardware, but in sys-net those values are remapped, causing firmware crashes.
Last symptoms in dmesg in sys-net (after giving it enough RAM):
[ 4.613162] ath11k_pci 0000:00:06.0: BAR 0 [mem 0xf2000000-0xf21fffff 64bit]: assigned
[ 4.619723] ath11k_pci 0000:00:06.0: MSI vectors: 1
[ 4.620195] ath11k_pci 0000:00:06.0: wcn6855 hw2.1
[ 5.600245] ath11k_pci 0000:00:06.0: chip_id 0x12 chip_family 0xb board_id 0xff soc_id 0x400c1211
[ 5.600658] ath11k_pci 0000:00:06.0: fw_version 0x11088c35 fw_build_timestamp 2024-04-17 08:34 fw_build_id WLAN.HSP.1.1-03125-QCAHSPSWPL_V1_V2_SILICONZ_LITE-3.6510.41
[ 5.724905] ath11k_pci 0000:00:06.0: leaving PCI ASPM disabled to avoid MHI M2 problems
[ 6.783575] ath11k_pci 0000:00:06.0: failed to receive control response completion, polling..
[ 7.807559] ath11k_pci 0000:00:06.0: Service connect timeout
[ 7.807595] ath11k_pci 0000:00:06.0: failed to connect to HTT: -110
[ 7.807927] ath11k_pci 0000:00:06.0: failed to start core: -110
[ 8.016769] ath11k_pci 0000:00:06.0: firmware crashed: MHI_CB_EE_RDDM
[ 8.016834] ath11k_pci 0000:00:06.0: ignore reset dev flags 0x4000
[ 8.124355] ath11k_pci 0000:00:06.0: firmware crashed: MHI_CB_EE_RDDM
[ 18.112565] ath11k_pci 0000:00:06.0: failed to wait wlan mode request (mode 4): -110
[ 18.112606] ath11k_pci 0000:00:06.0: qmi failed to send wlan mode off: -110
Solution
Applying an (unmerged) kernel patch to sys-net that adds module parameters to manually pass MSI values to the driver.
Please read the notes at the bottom for more information before following this.
1. Get MSI Values for the Network Card
In dom0:
sudo lspci -s 02:00.0 -vv | grep -A1 "MSI:"
Replace 02:00.0 with the ID of your network card shown in lspci | grep -i network. Note the values for āAddressā and āDataā. Mine were fee0a000 and 4000.
2. Create a Template for sys-net
If you donāt already have a dedicated template, clone the fedora-42-xfce template (I called it fedora-net-firmware) and update it with dnf update. Power off the template, make it a HVM, and set the kernel to (provided by qube). If you run uname -r in the template, you should see something like 6.17.13-200.fc42.x86_64 instead of 6.12.59-1.qubes.fc41.x86_64 (may be newer for you).
3. Create a StandaloneVM for Building the Patch
Create a StandaloneVM based on the new template, make it an HVM and set the kernel to (provided by qube) (I called it fedora-net-build). Give it 4 CPUs and 4GB of RAM (or however much RAM/CPUs are available) and extra storage (64 GB is probably enough).
In the new StandaloneVM:
dnf install -y gcc make flex bison openssl openssl-devel elfutils-libelf-devel \
perl ncurses-devel bc dwarves rsync git patch kernel-devel-$(uname -r)
4. Setup Build Environment
The 20GB cap on the root directory (which includes /usr/src/linux) will not be enough to build the kernel. To take advantage of the allocated storage, we create a symlink:
mkdir -p /rw/usr_src_linux
ln -s /rw/usr_src_linux /usr/src/linux
Now clone the kernel repository:
cd /usr/src
git clone --depth 1 --branch linux-6.17.y \
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
cd linux
Replace 6.17 with the series returned in uname -r.
Create ath11k-msi-workaround.patch:
--- a/drivers/net/wireless/ath/ath11k/pci.c
+++ b/drivers/net/wireless/ath/ath11k/pci.c
@@ -31,6 +31,14 @@
#define TCSR_SOC_HW_SUB_VER 0x1910010
+static unsigned long host_msi_vector_addr = 0;
+module_param(host_msi_vector_addr, ulong, 0644);
+MODULE_PARM_DESC(host_msi_vector_addr, "Host MSI vector address for VM passthrough");
+
+static unsigned long host_msi_vector_data = 0;
+module_param(host_msi_vector_data, ulong, 0644);
+MODULE_PARM_DESC(host_msi_vector_data, "Host MSI vector data for VM passthrough");
+
static const struct pci_device_id ath11k_pci_id_table[] = {
{ PCI_VDEVICE(QCOM, QCA6390_DEVICE_ID) },
{ PCI_VDEVICE(QCOM, WCN6855_DEVICE_ID) },
@@ -443,6 +451,18 @@ static int ath11k_pci_alloc_msi(struct ath11k_pci *ab_pci)
ath11k_pci_msi_disable(ab_pci);
+ if (host_msi_vector_addr && host_msi_vector_data) {
+ ab_pci->ab->pci.msi.ep_base_data = (u32)host_msi_vector_data;
+ ab->pci.msi.addr_hi = (u32)(host_msi_vector_addr >> 32);
+ ab->pci.msi.addr_lo = (u32)(host_msi_vector_addr & 0xffffffff);
+
+ ath11k_dbg(ab, ATH11K_DBG_PCI, "msi workaround: addr hi 0x%x lo 0x%x data %d\n",
+ ab->pci.msi.addr_hi,
+ ab->pci.msi.addr_lo,
+ ab->pci.msi.ep_base_data);
+ return 0;
+ }
+
msi_desc = irq_get_msi_desc(ab_pci->pdev->irq);
if (!msi_desc) {
ath11k_err(ab, "msi_desc is NULL!\n");
@@ -482,6 +502,9 @@ static int ath11k_pci_config_msi_data(struct ath11k_pci *ab_pci)
{
struct msi_desc *msi_desc;
+ if (host_msi_vector_addr && host_msi_vector_data)
+ return 0;
+
msi_desc = irq_get_msi_desc(ab_pci->pdev->irq);
if (!msi_desc) {
ath11k_err(ab_pci->ab, "msi_desc is NULL!\n");
This patch (originally tested on bhyve) was made by Jose Ignacio Tornos Martinez from Red Hat with the help of Baochen Qiang, an engineer at Qualcomm.
Try applying it:
patch -p1 --dry-run < ath11k-msi-workaround.patch
If that returns success, run it without --dry-run:
patch -p1 < ath11k-msi-workaround.patch
Prepare the source tree to build the kernel:
cp /boot/config-$(uname -r) .config
make olddefconfig
You can now build the kernel with the patch applied. This will take a while.
make -j$(nproc)
5. Build the Patched Modules
Now itās time to build a new set of patched modules. You may notice that ath11k.ko and ath11k_pci.ko are already compiled, but these need to match your kernel version (from uname -r) exactly.
cd drivers/net/wireless/ath/ath11k
make -C /lib/modules/$(uname -r)/build M=$(pwd) modules
To check the new modules, run:
modinfo ath11k_pci.ko | grep -iE "vermagic|parm"
You should see your precise kernel version under vermagic (not ending in -dirty) and parameters for host_msi_vector_addr and host_msi_vector_data.
6. Install in Template
Copy your new modules to the template for sys-net:
qvm-copy ath11k.ko ath11k_pci.ko
Now to install the new modules:
cp /home/user/QubesIncoming/fedora-net-build/ath11k*.ko /lib/modules/$(uname -r)/updates/
depmod -a
And shut down the template once they are installed:
poweroff
7. Configure sys-net
Boot sys-net with this template. Unload the existing ath11k drivers:
rmmod ath11k_pci
rmmod ath11k
Re-load them with your new MSI parameters:
modprobe ath11k
modprobe ath11k_pci host_msi_vector_addr=0xfee0a000 host_msi_vector_data=0x4000
Replace fee0a000 and 4000 with the values you recorded earlier.
Your Qualcomm WiFi card should now load, and you should be able to see to nearby networks in the NetworkManager applet.
To make this happen every time, add these lines to the end of /rw/config/rc.local in sys-net (must not be disposable):
# Wait for automatic (failed) load to initialize device
sleep 10
# Reload with MSI workaround
rmmod ath11k_pci
rmmod ath11k
sleep 1
modprobe ath11k
modprobe ath11k_pci host_msi_vector_addr=0xfee0a000 host_msi_vector_data=0x4000
Replace fee0a000 and 4000 with the values you recorded earlier. If rc.local did not exist previously, set a #!/bin/sh shebang as the first line and make it executable with chmod +x /rw/config/rc.local.
Notes
-
You will probably need to re-build these modules every time your templateās kernel updates.
-
In my case, letting the driver fail once (as demonstrated in step 7) before reloading with parameters was necessary. Setting the values with a
.conffile at/etc/modprobe.d/did not work, and blacklisting the drivers before loading them fresh caused the initialization to fail. -
This process was a result of ongoing troubleshooting, so some steps may be unnecessary. Specifically, Iām still not sure if compiling the whole kernel is necessary before building the modules (I had already tried compiling the modules with the upstream kernel before discovering I needed to match the build more precisely). Additionally, much of this report is coming from memory, so I may have missed some details.
-
Troubleshooting steps are available in the comments that may help.
If you have any suggestions to correct/improve this guide (especially if you have a better alternative to the hack on step 7) or report how it goes on your hardware on Qubes 4.3, I would be grateful.
