Windows VM BSOD (KMODE EXCEPTION NOT HANDLED) (IRQL NOT LESS OR EQUAL)

[sweet.cheddar]

Hello fellow qubes users,

I have tried for quite some time but i am not able to have a stable windows-vm.
I successfully can run windows10 and windows11 but after a while (it can be 2min or 45min) i run into a blue screen KMODE EXCEPTION NOT HANDLED in windows10 (+ continous beeping when browser is playing music) and windows11 and sometimes i got IRQL NOT LESS OR EQUAL in windows11.

I followed those steps :
First, I created a windows template following the steps described here :

How to install Windows qubes in Qubes OS

(After creation i set qvm-prefs WindowsNew qrexec_timeout 7200 and powercfg -H off in windows-vm)

Then I installed QWT following :

Qubes Windows Tools (QWT)

I tried with and without xen drivers - i seem to got the same result

After successfully install windows : I run a file check : sfc /scannow it usually repairs some files the first time, the next time i run it after a bsod and a reebot, it says its all ok.

I updated windows

I ran memory check, all is ok.

Before installing QWT, in device manager in windows, i have a yellow triangle in front of pci device.
I disabled the faulty driver in device manager in the windows-vm

Then i created an appvm based on that windows-vm.

So, i think it might be more of a hardware related issue…

when I run: journalctl -f -n0in dom0
I get a pcie bus error

The pcie bus error is probably due with my wifi card qualcomm atheros but the error is detected and corrected.
Similar to this issue : "PCIe Bus Error" for my Qualcomm Ethernet controller / Kernel & Hardware / Arch Linux Forums
So I suppressed this messages in case windows vm was having a hard time handling all those errors,
adding to dom0: /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash pci=noaer"
and saving changes
Now nothing appears when i run journalctl -f -n0 in dom0

The problem description i have is similar to this one Windows Bluescreen ( KMODE EXCEPTION NOT HANDLED ) - #20 by lie
But I don’t think the error come from the cpu, but i still tried to disable Split Lock Detection on the Linux host.

Windows-vm seems a bit more stable following those steps, but stills crashes.

Should i run tailf and monitor from dom0 to monitor as described in the qubes doc for installing windows ? if so i can’t find tailf nor the files to check.

Here are the results of xl dmesg
result-from-xl-dmesg.log (8.4 KB)

and the stubdom log for win10 and win11 located in /var/log/xen/console/guest-VMNAME-dm.log
windows10pro.log (54.3 KB)
windows11pro.log (54.8 KB)

Can some one please help me to understand if there is errors in here ?
And where my issue might be ?

[apparatus]

Do you have the same issue without QWT installed?

[sweet.cheddar]

Yes, i get the same BSOD without QWT.

Windows10 : Kmode Exception Not Handled
Windows11 : Kmode Exception Not Handled and IRQL Not Less or Equal

[sweet.cheddar]

For info i’m running qubes 4.2.2 but i had the same issue with this computer with qubes 4.1.

From an external ssd with debian, I tried running a windows-vm - with quickemu and virtualbox, it work, no bsod so far.
(I get the same pcie bus error (detected and corrected) in journalctl)

.

As a workaround, I am now trying to to convert VirtualBox VMs to Qubes HVMs as described here : Standalones and HVMs | Qubes OS

• I have a .vdi file and not a .vmdk
• I converted .vdi to raw with qemu : qemu-img convert -O raw win10.vdi win10.raw
• (I first tried to create a new windows-template and boot from the raw file - without success - no bootable devices found)
• i then copied the file to dom0 and created a new HVM as decribed - without success - vm is stuck for 15min at “booting from Hard Disk…”

Maybe something went wrong transfering the vdi file to qubes, i will try again later.

Don’t hesitate to give me a clue

[apparatus]

@sweet.cheddar

As a workaround, I am now trying to to convert VirtualBox VMs to Qubes HVMs as described here : Standalones and HVMs | Qubes OS

• I have a .vdi file and not a .vmdk
• I converted .vdi to raw with qemu : qemu-img convert -O raw win10.vdi win10.raw
• (I first tried to create a new windows-template and boot from the raw file - without success - no bootable devices found)
• i then copied the file to dom0 and created a new HVM as decribed - without success - vm is stuck for 15min at “booting from Hard Disk…”

Maybe something went wrong transfering the vdi file to qubes, i will try again later.

Don’t hesitate to give me a clue

Maybe you’ve installed Windows VM in UEFI mode. The Qubes OS is using Legacy BIOS boot mode by default.
You can enable UEFI for a qube by setting uefi feature for it:
qvm-features <vm-name> uefi 1
And you need to add ovmf.bin in your dom0:

UEFI enabled HVMs hang on "Guest has not initialized the display (yet)"

Boot from qube and if you end up in startup.nsh shell then enter exit to enter UEFI menu and in Boot Manger select WIndows boot menu entry.

[sweet.cheddar]

Thank you.

WindowsVM was installed in legacy bios.

The issue was that i was too quick and skiped steps.

[sweet.cheddar]

Workaround :

see : ‘Converting VirtualBox VMs to Qubes HVMs’ in Standalones and HVMs | Qubes OS

From an externall ssd connected to my laptop :

• Create a WindowsVM with VirtualBox
  Choose vdmk :
  Change preferences > ‘expert’ then you can chose vmdk
  (check Skip unattended install)
  (Be sure to not use EFI mode - disabled by default – as it can create additional issues see UEFI enabled HVMs hang on "Guest has not initialized the display (yet)" · Issue #8625 · QubesOS/qubes-issues · GitHub)

• Export vm :
  Select “File” > export appliance to .ova

From QubesOs :

• Copy your .ova file to a qubes

• Extract your OVA tar archive
  tar -xvf *.ova

• Convert vmdk to raw:
  (Increase size of qube if necessary to 60GB+)
  qemu-img convert -O raw *.vmdk win10.raw

• Copy the root image file from the originating qube (here called untrusted) to a temporary location in dom0, typing this in a dom0 terminal:
  (increase dom0 size if needed)
  qvm-run --pass-io untrusted 'cat "/media/user/externalhd/win10.raw"' > /home/user/win10-root.img

• From within dom0, create a new HVM (here called win10) with the root image we just copied to dom0 (change the amount of RAM in GB as you wish):
  (for a template change --standalone with --class TemplateVM)
  qvm-create --property=virt_mode=hvm --property=memory=4096 --property=kernel=‘’ --label red --standalone --root-move-from /home/user/win10-root.img win10

I now have a working Windows10VM, I also installed Qubes Windows Tools as described here : Qubes Windows Tools (QWT) | Qubes OS

No BSOD so far. QWT works.

[sweet.cheddar]

never mind i just get a BSOD kMode exception not handled in my new windowsTemplate

It’s starting to driving me crazy that i seem to be the only one to have this issue

[apparatus]

If it works in other hypervisors e.g. KVM then maybe it’s Xen-specific issue and you can try to ask this question in Xen matrix channel or mailing list:
Matrix - Xen Project
Mailing Lists - Xen Project
Maybe they’ll be able to help debugging it.

[sweet.cheddar]

So i tried asking to xen team but it didn’t solve my issue.

I also asked to the vendor of my laptop, NV41.
They told me to have a look at GitHub - ElliotKillick/qvm-create-windows-qube: Spin up new Windows qubes quickly, effortlessly and securely on Qubes OS.
They told me that in their latest test they “observed stability issues during the install if it was finished successfully it mostly worked.”

I followed every step. Including downloading the iso with mido.

Sometimes when running the script for the creation of a new winvm, i get a bsod (kmode exception not handled), so i delete the newly created qube and start again.

Windows vm can be created with this script but they don’t have internet connectivity.

The “qvm-create-windows-qube” script is stuck in “Completing setup of Qubes Windows Tools…” and can’t go further.

Newly created winvm still get bsod after a while.

[tempmail]

@sweet.cheddar

QWT works.

Wait. Are you telling that you could fire up Win10 HVM with QWT installed under Qubes v4.2x, only sometimes later while it’s up to get BSOD?

[sweet.cheddar]

yes.

I got this bsod with and without QWT somewhere betwwen 2 minutes to maybe an hour after succesfully starting the windows HVM

[tempmail]

Wow. were you aware that you were the only one who succeeded to fire it up at all with qwt?

QWT (support for Windows in Qubes OS) is not available anymore. When will this be solved?

@marmarek says he will try to find the time to make QWT available with the final 4.2 release, but no guarantees! Sounds like the Qubes devs will have to set up their own infrastructure to build properly-signed Windows PV driver binaries themselves in order to reach this goal.

Win 7 and Win10/11 under Qubes 4.2

What makes Win7 qube being able to get external storage drive attached in 4.2 (meaning Xen disk drive being able to be installed and qube still bootable), but not for Win10/11. In all three I have qwt 4.1.68.1-1 installed.

Windows 4.2 Support Functionally Incomplete - User Impacts and Solutions

Continuing the discussion from Qubes OS 4.1 has reached end-of-life; extended security support continues until 2024-07-31: As a Windows user who needs Qubes as the (currently only?) way to run Windows 7, 10, and 11 systems in a reasonably secure manner, I have the impression that putting R4.1 to EOL is premature. Looking from this perspective, R4.2 is still not functionally completed, as long as the issues listed in What would you like to see improved in Qubes OS? are still open. As described …

Workaround to Attach Block and USB Devices to Win Qube(s) Under 4.2

I have found never uglier way to achieve some goal. IF YOU ARE WILLING TO RISK AND MESS WITH POTENTIALLY COMPROMISED XEN WI(N)DOWS PV DRIVERS, here’s how to do it. Create Win HVM. Downgrade dom0 to qwt 4.1.69-1. Start Win qube for QWT installation (qvm-start win-qube --install-windows-tools). Install qwt as usual in Win qube and DO NOT install network and disk drivers. Copy .msi from CD-ROM to qube if you’ll to take step 7. in this qube. Shutdown and start Win qube. Using specific program, li…

How much are Windows users important to devs actually

It’s been a year from this statement (August 23, 2023). After his Google account has been hacked causing cross-account damages, I have a user who’d be finally willing to transition his small business (up to 10 people) to Qubes, but now with 4.2 he wouldn’t be able to use Win S-HVM’s. He finally clearly understood Qubes benefits and agreed, but he can’t run his business without Win qube(s), simply because he has to run business software that is developed only for Windows. I am genuinely baffle…

Windows tools for 4.2?

Is there anyway to get Windows tools working on 4.2? Ref: Qubes Windows Tools (QWT) | Qubes OS

[sweet.cheddar]

I don’t think i am the only one.

I just followed the instructions, downgrading qwt to 4.1.69

Qubes Windows Tools (QWT)

[tempmail]

I have never seen anyone else, but regardless, if you read those topics, you probably realized that no Win10/11 qube will work with qwt installed under Qubes v4.2., and there is ugly, but the only one workaround for that.

[apparatus]

QWT installer works fine for me in Qubes OS 4.2 so it works for at least 2 people.

[tempmail]

That’s great to hear! Can you attach block devices, not as USB devices?

[apparatus]

Is block device attachment with Xen PV Disk Drivers the only thing that is not working for you in QWT? Or nothing from QWT works at all (clipboard/file move/copy/etc)?
There seems to be an issue with Xen PV Disk Drivers and btrfs:

Disable direct IO on Btrfs based storage

I’ve tried attaching the block device to Windows qube with latest Xen drivers (not from QWT) and it worked before:

Workaround to Attach Block and USB Devices to Win Qube(s) Under 4.2

The v9 Xen PV drivers works the same way as QWT PV drivers for me in Windows 10. Enable testsigning in WIndows by running this command in admin console: bcdedit /set testsigning on Restart qube. Install xenbus and xenvbd drivers. Restart qube. In some Linux disposable qube run these commands to create test block device: dd if=/dev/zero of=tst bs=1M count=256 sudo losetup -f tst Attach this block device to Windows qube. Open Disk Management and initialize/format the disk. Create some te…

But the result could be inconsistent because of the issue with btrfs.

[tempmail]

@apparatus

Is block device attachment with Xen PV Disk Drivers the only thing that is not working for you in QWT?

No. I can’t boot when qwt disk drivers are installed (any version, of any provider, not Qubes only). When I don’t install disk drivers, everything else works: USB attaching, file/commands manipulation, everything.

I’ll look into that issue though, thanks.

[tempmail]

@apparatus

https://github.com/QubesOS/qubes-issues/issues/9488https://github.com/QubesOS/qubes-issues/issues/9488

Well, it looks like it was finally traced. What I’m not sure at the first glance, can I manually disable direct IO, or will have to wait until new version of qubes-linux-utils?

Also, you don’t have btrfs? Cause I do.