Win10 VM now trying to boot from USB controller?

slcoleman · September 24, 2022, 4:14pm

I have a Win10 VM that used to work just fine but no longer boots after the recent updates. I have restored from backup almost every version going back in history and none of them boot any more, even from backups that I previously restored from before and know that they worked just fine. Something in Qubes/Xen must have changed recently so that Qubes is not finding the proper Windows boot record/partition.

What I see during boot is the window frame opens and display’s the following:

SeaBIOS (version 1.13.0-3.fc32)
Machine UUID XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
Booting from Hard Disk…
Boot failed: could not read the boot disk

Booting from Floppy…
Boot failed: could not read the boot disk

No bootable device.

All known good backups produced the exact same result going back as much as a year. What did finally allow the VM to boot is removing its USB controller board that used to work just fine. That controller is required to do any scanning/faxing/printing from the Windows VM where the software is only provided for Windows based OS’s, and that software is too stupid to “locate” the printer on the locally attached network. Apparently it doesn’t look beyond the Qubes internal network for a printer.

The device:

71:00.0 USB controller: NEC Corporation uPD720200 USB 3.0 Host Controller

My guess is that Xen/Qubes is now trying to boot that Win10 VM from that USB controller board instead. This behaviour apparently changed in the past few months. I remember there being some Xen updates in the recent past and I had not tried running this VM since those updates.

Q1: What changed in the last set of Xen/Dom0 updates that might affect this?

Q2: Are there any options that will force the VM to boot from the proper volume by hiding the controller while the VM is booting up but allow it to be useable after its booted?

guest-Win10-dm.log:

[2022-09-24 10:45:17] Logfile Opened
[2022-09-24 10:45:18] Linux version 5.10.105-xen-stubdom (mockbuild@0c9ad8f2058f40c49bc934dcc2ea73c7) (gcc (GCC) 10.3.1 20210422 (Red Hat 10.3.1-1), GNU ld version 2.34-6.fc32) #1 Fri Apr 22 17:47:07 CEST 2022
[2022-09-24 10:45:18] Command line:
[2022-09-24 10:45:18] x86/fpu: Supporting XSAVE feature 0x001: ‘x87 floating point registers’
[2022-09-24 10:45:18] x86/fpu: Supporting XSAVE feature 0x002: ‘SSE registers’
[2022-09-24 10:45:18] x86/fpu: Supporting XSAVE feature 0x004: ‘AVX registers’
[2022-09-24 10:45:18] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256
[2022-09-24 10:45:18] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using ‘standard’ format.
[2022-09-24 10:45:18] Released 0 page(s)
[2022-09-24 10:45:18] BIOS-provided physical RAM map:
…
[2022-09-24 10:45:18] tsc: Fast TSC calibration failed
[2022-09-24 10:45:18] tsc: Detected 3599.998 MHz processor
[2022-09-24 10:45:18] last_pfn = 0x9000 max_arch_pfn = 0x400000000
[2022-09-24 10:45:18] x86/PAT: Configuration [0-7]: WB WT UC- UC WC WP UC UC
…
[2022-09-24 10:45:18] Mount-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
[2022-09-24 10:45:18] Mountpoint-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
[2022-09-24 10:45:18] PCI: setting up Xen PCI frontend stub
[2022-09-24 10:45:18] xen:balloon: Initialising balloon driver
[2022-09-24 10:45:18] PCI: System does not support PCI
…
[2022-09-24 10:45:18] Unpacking initramfs…
[2022-09-24 10:45:18] Freeing initrd memory: 27752K
[2022-09-24 10:45:18] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x33e451ab1a6, max_idle_ns: 440795278720 ns
[2022-09-24 10:45:18] clocksource: Switched to clocksource tsc
[2022-09-24 10:45:18] workingset: timestamp_bits=62 max_order=15 bucket_order=0
[2022-09-24 10:45:18] xen:xen_evtchn: Event-channel device installed
[2022-09-24 10:45:18] Invalid max_queues (4), will use default max: 1.
[2022-09-24 10:45:18] tun: Universal TUN/TAP device driver, 1.6
[2022-09-24 10:45:18] xen_netfront: Initialising Xen virtual ethernet driver
[2022-09-24 10:45:18] pcifront pci-0: Installing PCI frontend
[2022-09-24 10:45:18] xen:swiotlb_xen: Warning: only able to allocate 4 MB for software IO TLB
[2022-09-24 10:45:18] software IO TLB: mapped [mem 0x0000000003400000-0x0000000003800000] (4MB)
[2022-09-24 10:45:18] NET: Registered protocol family 17
[2022-09-24 10:45:18] sched_clock: Marking stable (156869322, 138846)->(159968338, -2960170)
[2022-09-24 10:45:18] pcifront pci-0: Creating PCI Frontend Bus 0000:00
[2022-09-24 10:45:18] pcifront pci-0: PCI host bridge to bus 0000:00
[2022-09-24 10:45:18] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]
[2022-09-24 10:45:18] pci_bus 0000:00: root bus resource [mem 0x00000000-0x7fffffffff]
[2022-09-24 10:45:18] pci_bus 0000:00: root bus resource [bus 00-ff]
[2022-09-24 10:45:18] pci 0000:00:00.0: [1033:0194] type 00 class 0x0c0330
[2022-09-24 10:45:18] pci 0000:00:00.0: reg 0x10: [mem 0xee400000-0xee401fff 64bit]
[2022-09-24 10:45:18] pcifront pci-0: claiming resource 0000:00:00.0/0
[2022-09-24 10:45:18] random: fast init done
[2022-09-24 10:45:18] xen_netfront: backend supports XDP headroom
[2022-09-24 10:45:18] blkfront: xvda: flush diskcache: enabled; persistent grants: enabled; indirect descriptors: enabled;
[2022-09-24 10:45:18] xvda: xvda1
[2022-09-24 10:45:18] blkfront: xvdb: flush diskcache: enabled; persistent grants: enabled; indirect descriptors: enabled;
[2022-09-24 10:45:18] xvdb: xvdb1
[2022-09-24 10:45:18] blkfront: xvdc: flush diskcache: enabled; persistent grants: enabled; indirect descriptors: enabled;
[2022-09-24 10:45:18] Freeing unused kernel image (initmem) memory: 696K
[2022-09-24 10:45:18] Write protecting the kernel read-only data: 10240k
[2022-09-24 10:45:18] Freeing unused kernel image (text/rodata gap) memory: 2044K
[2022-09-24 10:45:18] Freeing unused kernel image (rodata/data gap) memory: 1648K
[2022-09-24 10:45:18] Run /init as init process
[2022-09-24 10:45:18] + mount -t devtmpfs none /dev
[2022-09-24 10:45:18] + mount -t sysfs /sys /sys
[2022-09-24 10:45:18] + mount -t proc /proc /proc
[2022-09-24 10:45:18] + mount -t tmpfs -o ‘size=1m,nodev,noexec’ /tmp /tmp
[2022-09-24 10:45:18] + mount -o remount,ro /
[2022-09-24 10:45:18] + echo 1
[2022-09-24 10:45:18] + printf ‘%d\n’ 1073741824
[2022-09-24 10:45:18] + /bin/xenstore-read target
[2022-09-24 10:45:18] + domid=46
[2022-09-24 10:45:18] + xenstore-read /local/domain/46/vm
[2022-09-24 10:45:18] + vm_path=/vm/b39d291b-058a-4585-8c74-acfcf8d4856b
[2022-09-24 10:45:18] + sort
[2022-09-24 10:45:18] + xenstore-list -p /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv
[2022-09-24 10:45:18] + xenstore-read /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/001 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/002 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/003 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/004 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/005 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/006 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/007 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/008 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/009 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/010 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/011 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/012 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/013 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/014 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/015 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/016 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/017 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/018 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/019 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/020 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/021 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/022 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/023 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/024 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/025 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/026 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/027 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/028 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/029 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/030 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/031 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/032 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/033 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/034 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/035 /vm/b39d291b-058a-4585-8c74-acfcf8d4856b/image/dm-argv/036
[2022-09-24 10:45:18] + dm_args=‘-xen-domid
[2022-09-24 10:45:18] 46
[2022-09-24 10:45:18] -no-shutdown
[2022-09-24 10:45:18] -nodefaults
[2022-09-24 10:45:18] -no-user-config
[2022-09-24 10:45:18] -name
[2022-09-24 10:45:18] Win10
[2022-09-24 10:45:18] -display
[2022-09-24 10:45:18] none
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] VGA,vgamem_mb=16
[2022-09-24 10:45:18] -boot
[2022-09-24 10:45:18] order=dc
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] usb-ehci,id=ehci
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] usb-tablet,bus=ehci.0
[2022-09-24 10:45:18] -smp
[2022-09-24 10:45:18] 4,maxcpus=4
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] rtl8139,id=nic0,netdev=net0,mac=00:16:3e:5e:6c:00
[2022-09-24 10:45:18] -netdev
[2022-09-24 10:45:18] type=tap,id=net0,ifname=vif46.0-emu,script=no,downscript=no
[2022-09-24 10:45:18] -display
[2022-09-24 10:45:18] qubes-gui,domid=0,log-level=0
[2022-09-24 10:45:18] -machine
[2022-09-24 10:45:18] xenfv
[2022-09-24 10:45:18] -qubes-net:client_ip=10.137.0.1,dns_0=10.139.1.1,dns_1=10.139.1.2,gw=10.137.0.7,netmask=255.255.255.255
[2022-09-24 10:45:18] -m
[2022-09-24 10:45:18] 7984
[2022-09-24 10:45:18] -drive
[2022-09-24 10:45:18] file=/dev/xvda,if=ide,index=0,media=disk,format=host_device,cache=writeback
[2022-09-24 10:45:18] -drive
[2022-09-24 10:45:18] file=/dev/xvdb,if=ide,index=1,media=disk,format=host_device,cache=writeback
[2022-09-24 10:45:18] -drive
[2022-09-24 10:45:18] file=/dev/xvdc,if=ide,index=2,media=disk,format=host_device,cache=writeback’
[2022-09-24 10:45:18] + usb_args=
[2022-09-24 10:45:18] + test -e /bin/qrexec-agent
[2022-09-24 10:45:18] + sed -n ‘/^-soundhw/ {n;p}’
[2022-09-24 10:45:18] + echo ‘-xen-domid
[2022-09-24 10:45:18] 46
[2022-09-24 10:45:18] -no-shutdown
[2022-09-24 10:45:18] -nodefaults
[2022-09-24 10:45:18] -no-user-config
[2022-09-24 10:45:18] -name
[2022-09-24 10:45:18] Win10
[2022-09-24 10:45:18] -display
[2022-09-24 10:45:18] none
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] VGA,vgamem_mb=16
[2022-09-24 10:45:18] -boot
[2022-09-24 10:45:18] order=dc
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] usb-ehci,id=ehci
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] usb-tablet,bus=ehci.0
[2022-09-24 10:45:18] -smp
[2022-09-24 10:45:18] 4,maxcpus=4
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] rtl8139,id=nic0,netdev=net0,mac=00:16:3e:5e:6c:00
[2022-09-24 10:45:18] -netdev
[2022-09-24 10:45:18] type=tap,id=net0,ifname=vif46.0-emu,script=no,downscript=no
[2022-09-24 10:45:18] -display
[2022-09-24 10:45:18] qubes-gui,domid=0,log-level=0
[2022-09-24 10:45:18] -machine
[2022-09-24 10:45:18] xenfv
[2022-09-24 10:45:18] -qubes-net:client_ip=10.137.0.1,dns_0=10.139.1.1,dns_1=10.139.1.2,gw=10.137.0.7,netmask=255.255.255.255
[2022-09-24 10:45:18] -m
[2022-09-24 10:45:18] 7984
[2022-09-24 10:45:18] -drive
[2022-09-24 10:45:18] file=/dev/xvda,if=ide,index=0,media=disk,format=host_device,cache=writeback
[2022-09-24 10:45:18] -drive
[2022-09-24 10:45:18] file=/dev/xvdb,if=ide,index=1,media=disk,format=host_device,cache=writeback
[2022-09-24 10:45:18] -drive
[2022-09-24 10:45:18] file=/dev/xvdc,if=ide,index=2,media=disk,format=host_device,cache=writeback’
[2022-09-24 10:45:18] + audio_model=
[2022-09-24 10:45:18] + ‘[’ -n ]
[2022-09-24 10:45:18] + sed -n /^-qubes-net:/p
[2022-09-24 10:45:18] + echo ‘-xen-domid
[2022-09-24 10:45:18] 46
[2022-09-24 10:45:18] -no-shutdown
[2022-09-24 10:45:18] -nodefaults
[2022-09-24 10:45:18] -no-user-config
[2022-09-24 10:45:18] -name
[2022-09-24 10:45:18] Win10
[2022-09-24 10:45:18] -display
[2022-09-24 10:45:18] none
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] VGA,vgamem_mb=16
[2022-09-24 10:45:18] -boot
[2022-09-24 10:45:18] order=dc
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] usb-ehci,id=ehci
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] usb-tablet,bus=ehci.0
[2022-09-24 10:45:18] -smp
[2022-09-24 10:45:18] 4,maxcpus=4
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] rtl8139,id=nic0,netdev=net0,mac=00:16:3e:5e:6c:00
[2022-09-24 10:45:18] -netdev
[2022-09-24 10:45:18] type=tap,id=net0,ifname=vif46.0-emu,script=no,downscript=no
[2022-09-24 10:45:18] -display
[2022-09-24 10:45:18] qubes-gui,domid=0,log-level=0
[2022-09-24 10:45:18] -machine
[2022-09-24 10:45:18] xenfv
[2022-09-24 10:45:18] -qubes-net:client_ip=10.137.0.1,dns_0=10.139.1.1,dns_1=10.139.1.2,gw=10.137.0.7,netmask=255.255.255.255
[2022-09-24 10:45:18] -m
[2022-09-24 10:45:18] 7984
[2022-09-24 10:45:18] -drive
[2022-09-24 10:45:18] file=/dev/xvda,if=ide,index=0,media=disk,format=host_device,cache=writeback
[2022-09-24 10:45:18] -drive
[2022-09-24 10:45:18] file=/dev/xvdb,if=ide,index=1,media=disk,format=host_device,cache=writeback
[2022-09-24 10:45:18] -drive
[2022-09-24 10:45:18] file=/dev/xvdc,if=ide,index=2,media=disk,format=host_device,cache=writeback’
[2022-09-24 10:45:18] + net_args=‘-qubes-net:client_ip=10.137.0.1,dns_0=10.139.1.1,dns_1=10.139.1.2,gw=10.137.0.7,netmask=255.255.255.255’
[2022-09-24 10:45:18] + sed /^-qubes-net:/d
[2022-09-24 10:45:18] + echo ‘-xen-domid
[2022-09-24 10:45:18] 46
[2022-09-24 10:45:18] -no-shutdown
[2022-09-24 10:45:18] -nodefaults
[2022-09-24 10:45:18] -no-user-config
[2022-09-24 10:45:18] -name
[2022-09-24 10:45:18] Win10
[2022-09-24 10:45:18] -display
[2022-09-24 10:45:18] none
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] VGA,vgamem_mb=16
[2022-09-24 10:45:18] -boot
[2022-09-24 10:45:18] order=dc
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] usb-ehci,id=ehci
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] usb-tablet,bus=ehci.0
[2022-09-24 10:45:18] -smp
[2022-09-24 10:45:18] 4,maxcpus=4
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] rtl8139,id=nic0,netdev=net0,mac=00:16:3e:5e:6c:00
[2022-09-24 10:45:18] -netdev
[2022-09-24 10:45:18] type=tap,id=net0,ifname=vif46.0-emu,script=no,downscript=no
[2022-09-24 10:45:18] -display
[2022-09-24 10:45:18] qubes-gui,domid=0,log-level=0
[2022-09-24 10:45:18] -machine
[2022-09-24 10:45:18] xenfv
[2022-09-24 10:45:18] -qubes-net:client_ip=10.137.0.1,dns_0=10.139.1.1,dns_1=10.139.1.2,gw=10.137.0.7,netmask=255.255.255.255
[2022-09-24 10:45:18] -m
[2022-09-24 10:45:18] 7984
[2022-09-24 10:45:18] -drive
[2022-09-24 10:45:18] file=/dev/xvda,if=ide,index=0,media=disk,format=host_device,cache=writeback
[2022-09-24 10:45:18] -drive
[2022-09-24 10:45:18] file=/dev/xvdb,if=ide,index=1,media=disk,format=host_device,cache=writeback
[2022-09-24 10:45:18] -drive
[2022-09-24 10:45:18] file=/dev/xvdc,if=ide,index=2,media=disk,format=host_device,cache=writeback’
[2022-09-24 10:45:18] + dm_args=‘-xen-domid
[2022-09-24 10:45:18] 46
[2022-09-24 10:45:18] -no-shutdown
[2022-09-24 10:45:18] -nodefaults
[2022-09-24 10:45:18] -no-user-config
[2022-09-24 10:45:18] -name
[2022-09-24 10:45:18] Win10
[2022-09-24 10:45:18] -display
[2022-09-24 10:45:18] none
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] VGA,vgamem_mb=16
[2022-09-24 10:45:18] -boot
[2022-09-24 10:45:18] order=dc
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] usb-ehci,id=ehci
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] usb-tablet,bus=ehci.0
[2022-09-24 10:45:18] -smp
[2022-09-24 10:45:18] 4,maxcpus=4
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] rtl8139,id=nic0,netdev=net0,mac=00:16:3e:5e:6c:00
[2022-09-24 10:45:18] -netdev
[2022-09-24 10:45:18] type=tap,id=net0,ifname=vif46.0-emu,script=no,downscript=no
[2022-09-24 10:45:18] -display
[2022-09-24 10:45:18] qubes-gui,domid=0,log-level=0
[2022-09-24 10:45:18] -machine
[2022-09-24 10:45:18] xenfv
[2022-09-24 10:45:18] -m
[2022-09-24 10:45:18] 7984
[2022-09-24 10:45:18] -drive
[2022-09-24 10:45:18] file=/dev/xvda,if=ide,index=0,media=disk,format=host_device,cache=writeback
[2022-09-24 10:45:18] -drive
[2022-09-24 10:45:18] file=/dev/xvdb,if=ide,index=1,media=disk,format=host_device,cache=writeback
[2022-09-24 10:45:18] -drive
[2022-09-24 10:45:18] file=/dev/xvdc,if=ide,index=2,media=disk,format=host_device,cache=writeback’
[2022-09-24 10:45:18] + test -e /sys/class/net/eth0
[2022-09-24 10:45:18] + ip link set eth0 address fe:ff:ff:ff:ff:fe
[2022-09-24 10:45:18] + ip addr flush eth0
[2022-09-24 10:45:18] + ip link set eth0 up
[2022-09-24 10:45:18] + brctl addbr br0
[2022-09-24 10:45:18] + brctl addif br0 eth0
[2022-09-24 10:45:18] br0: port 1(eth0) entered blocking state
[2022-09-24 10:45:18] br0: port 1(eth0) entered disabled state
[2022-09-24 10:45:18] device eth0 entered promiscuous mode
[2022-09-24 10:45:18] + ip link set br0 up
[2022-09-24 10:45:18] br0: port 1(eth0) entered blocking state
[2022-09-24 10:45:18] br0: port 1(eth0) entered forwarding state
[2022-09-24 10:45:18] + get_net_arg dns_0
[2022-09-24 10:45:18] + sed -n ‘s/^.[:,]dns_0=([^,]+).$/\1/p’
[2022-09-24 10:45:18] + echo ‘-qubes-net:client_ip=10.137.0.1,dns_0=10.139.1.1,dns_1=10.139.1.2,gw=10.137.0.7,netmask=255.255.255.255’
[2022-09-24 10:45:18] + net_dns_0=10.139.1.1
[2022-09-24 10:45:18] + get_net_arg dns_1
[2022-09-24 10:45:18] + sed -n ‘s/^.[:,]dns_1=([^,]+).$/\1/p’
[2022-09-24 10:45:18] + echo ‘-qubes-net:client_ip=10.137.0.1,dns_0=10.139.1.1,dns_1=10.139.1.2,gw=10.137.0.7,netmask=255.255.255.255’
[2022-09-24 10:45:18] + net_dns_1=10.139.1.2
[2022-09-24 10:45:18] + get_net_arg client_ip
[2022-09-24 10:45:18] + sed -n ‘s/^.[:,]client_ip=([^,]+).$/\1/p’
[2022-09-24 10:45:18] + echo ‘-qubes-net:client_ip=10.137.0.1,dns_0=10.139.1.1,dns_1=10.139.1.2,gw=10.137.0.7,netmask=255.255.255.255’
[2022-09-24 10:45:18] + net_client_ip=10.137.0.1
[2022-09-24 10:45:18] + get_net_arg netmask
[2022-09-24 10:45:18] + sed -n ‘s/^.[:,]netmask=([^,]+).$/\1/p’
[2022-09-24 10:45:18] + echo ‘-qubes-net:client_ip=10.137.0.1,dns_0=10.139.1.1,dns_1=10.139.1.2,gw=10.137.0.7,netmask=255.255.255.255’
[2022-09-24 10:45:18] + net_netmask=255.255.255.255
[2022-09-24 10:45:18] + get_net_arg gw
[2022-09-24 10:45:18] + sed -n ‘s/^.[:,]gw=([^,]+).$/\1/p’
[2022-09-24 10:45:18] + echo ‘-qubes-net:client_ip=10.137.0.1,dns_0=10.139.1.1,dns_1=10.139.1.2,gw=10.137.0.7,netmask=255.255.255.255’
[2022-09-24 10:45:18] + net_gw=10.137.0.7
[2022-09-24 10:45:18] + xenstore-read target
[2022-09-24 10:45:18] + udhcpd -f -I 10.137.0.7 -
[2022-09-24 10:45:18] + target=46
[2022-09-24 10:45:18] + device_model=device-model/46
[2022-09-24 10:45:18] + mkdir /tmp/qmp
[2022-09-24 10:45:18] + kernel=
[2022-09-24 10:45:18] udhcpd: started, v1.31.1
[2022-09-24 10:45:18] + grep -q ^-append
[2022-09-24 10:45:18] + echo ‘-xen-domid
[2022-09-24 10:45:18] 46
[2022-09-24 10:45:18] -no-shutdown
[2022-09-24 10:45:18] -nodefaults
[2022-09-24 10:45:18] -no-user-config
[2022-09-24 10:45:18] -name
[2022-09-24 10:45:18] Win10
[2022-09-24 10:45:18] -display
[2022-09-24 10:45:18] none
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] VGA,vgamem_mb=16
[2022-09-24 10:45:18] -boot
[2022-09-24 10:45:18] order=dc
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] usb-ehci,id=ehci
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] usb-tablet,bus=ehci.0
[2022-09-24 10:45:18] -smp
[2022-09-24 10:45:18] 4,maxcpus=4
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] rtl8139,id=nic0,netdev=net0,mac=00:16:3e:5e:6c:00
[2022-09-24 10:45:18] -netdev
[2022-09-24 10:45:18] type=tap,id=net0,ifname=vif46.0-emu,script=no,downscript=no
[2022-09-24 10:45:18] -display
[2022-09-24 10:45:18] qubes-gui,domid=0,log-level=0
[2022-09-24 10:45:18] -machine
[2022-09-24 10:45:18] xenfv
[2022-09-24 10:45:18] -m
[2022-09-24 10:45:18] 7984
[2022-09-24 10:45:18] -drive
[2022-09-24 10:45:18] file=/dev/xvda,if=ide,index=0,media=disk,format=host_device,cache=writeback
[2022-09-24 10:45:18] -drive
[2022-09-24 10:45:18] file=/dev/xvdb,if=ide,index=1,media=disk,format=host_device,cache=writeback
[2022-09-24 10:45:18] -drive
[2022-09-24 10:45:18] file=/dev/xvdc,if=ide,index=2,media=disk,format=host_device,cache=writeback’
[2022-09-24 10:45:18] + mkfifo /tmp/qmp/qemu.in /tmp/qmp/qemu.out
[2022-09-24 10:45:18] + set +x
[2022-09-24 10:45:18] Clearing kmsg buffer…
[2022-09-24 10:45:18] + set +x
[2022-09-24 10:45:18] + set +x
[2022-09-24 10:45:18] + sed ‘s/$STUBDOM_RESTORE_INCOMING_ARG/fd:3/’
[2022-09-24 10:45:18] + echo ‘-xen-domid
[2022-09-24 10:45:18] 46
[2022-09-24 10:45:18] -no-shutdown
[2022-09-24 10:45:18] -nodefaults
[2022-09-24 10:45:18] -no-user-config
[2022-09-24 10:45:18] -name
[2022-09-24 10:45:18] Win10
[2022-09-24 10:45:18] -display
[2022-09-24 10:45:18] none
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] VGA,vgamem_mb=16
[2022-09-24 10:45:18] -boot
[2022-09-24 10:45:18] order=dc
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] usb-ehci,id=ehci
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] usb-tablet,bus=ehci.0
[2022-09-24 10:45:18] -smp
[2022-09-24 10:45:18] 4,maxcpus=4
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] rtl8139,id=nic0,netdev=net0,mac=00:16:3e:5e:6c:00
[2022-09-24 10:45:18] -netdev
[2022-09-24 10:45:18] type=tap,id=net0,ifname=vif46.0-emu,script=no,downscript=no
[2022-09-24 10:45:18] -display
[2022-09-24 10:45:18] qubes-gui,domid=0,log-level=0
[2022-09-24 10:45:18] -machine
[2022-09-24 10:45:18] xenfv
[2022-09-24 10:45:18] -m
[2022-09-24 10:45:18] 7984
[2022-09-24 10:45:18] -drive
[2022-09-24 10:45:18] file=/dev/xvda,if=ide,index=0,media=disk,format=host_device,cache=writeback
[2022-09-24 10:45:18] -drive
[2022-09-24 10:45:18] file=/dev/xvdb,if=ide,index=1,media=disk,format=host_device,cache=writeback
[2022-09-24 10:45:18] -drive
[2022-09-24 10:45:18] file=/dev/xvdc,if=ide,index=2,media=disk,format=host_device,cache=writeback’
[2022-09-24 10:45:18] + dm_args=‘-xen-domid
[2022-09-24 10:45:18] 46
[2022-09-24 10:45:18] -no-shutdown
[2022-09-24 10:45:18] -nodefaults
[2022-09-24 10:45:18] -no-user-config
[2022-09-24 10:45:18] -name
[2022-09-24 10:45:18] Win10
[2022-09-24 10:45:18] -display
[2022-09-24 10:45:18] none
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] VGA,vgamem_mb=16
[2022-09-24 10:45:18] -boot
[2022-09-24 10:45:18] order=dc
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] usb-ehci,id=ehci
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] usb-tablet,bus=ehci.0
[2022-09-24 10:45:18] -smp
[2022-09-24 10:45:18] 4,maxcpus=4
[2022-09-24 10:45:18] -device
[2022-09-24 10:45:18] rtl8139,id=nic0,netdev=net0,mac=00:16:3e:5e:6c:00
[2022-09-24 10:45:18] -netdev
[2022-09-24 10:45:18] type=tap,id=net0,ifname=vif46.0-emu,script=no,downscript=no
[2022-09-24 10:45:18] -display
[2022-09-24 10:45:18] qubes-gui,domid=0,log-level=0
[2022-09-24 10:45:18] -machine
[2022-09-24 10:45:18] xenfv
[2022-09-24 10:45:18] -m
[2022-09-24 10:45:18] 7984
[2022-09-24 10:45:18] -drive
[2022-09-24 10:45:18] file=/dev/xvda,if=ide,index=0,media=disk,format=host_device,cache=writeback
[2022-09-24 10:45:18] -drive
[2022-09-24 10:45:18] file=/dev/xvdb,if=ide,index=1,media=disk,format=host_device,cache=writeback
[2022-09-24 10:45:18] -drive
[2022-09-24 10:45:18] file=/dev/xvdc,if=ide,index=2,media=disk,format=host_device,cache=writeback’
[2022-09-24 10:45:18] + xenstore-read device/console/2
[2022-09-24 10:45:18] + xenstore-read device/console/1
[2022-09-24 10:45:18] + IFS=’
[2022-09-24 10:45:18] ’
[2022-09-24 10:45:18] + set -f
[2022-09-24 10:45:18] + set +f
[2022-09-24 10:45:18] + unset IFS
[2022-09-24 10:45:18] + qemu_pid=92
[2022-09-24 10:45:18] + ‘[’ ‘!’ -e /tmp/qemu.qmp ]
[2022-09-24 10:45:18] + sleep 0.1
[2022-09-24 10:45:18] + qemu -sandbox ‘on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny’ -chardev ‘pipe,path=/tmp/qmp/qemu,id=m’ -mon ‘chardev=m,mode=control’ -chardev ‘socket,server,nowait,path=/tmp/qemu.qmp,id=m2’ -mon ‘chardev=m2,mode=control’ -xen-domid 46 -no-shutdown -nodefaults -no-user-config -name Win10 -display none -device ‘VGA,vgamem_mb=16’ -boot ‘order=dc’ -device ‘usb-ehci,id=ehci’ -device ‘usb-tablet,bus=ehci.0’ -smp ‘4,maxcpus=4’ -device ‘rtl8139,id=nic0,netdev=net0,mac=00:16:3e:5e:6c:00’ -netdev ‘type=tap,id=net0,ifname=vif46.0-emu,script=no,downscript=no’ -display ‘qubes-gui,domid=0,log-level=0’ -machine xenfv -m 7984 -drive ‘file=/dev/xvda,if=ide,index=0,media=disk,format=host_device,cache=writeback’ -drive ‘file=/dev/xvdb,if=ide,index=1,media=disk,format=host_device,cache=writeback’ -drive ‘file=/dev/xvdc,if=ide,index=2,media=disk,format=host_device,cache=writeback’

[2022-09-24 10:45:18] Please use wait=off instead
[2022-09-24 10:45:18] + tee /proc/self/fd/2
[2022-09-24 10:45:18] + echo ‘{“execute”: “qmp_capabilities”}’
[2022-09-24 10:45:18] {“execute”: “qmp_capabilities”}
[2022-09-24 10:45:18] random: qemu: uninitialized urandom read (16 bytes read)
[2022-09-24 10:45:18] {“QMP”: {“version”: {“qemu”: {“micro”: 0, “minor”: 1, “major”: 6}, “package”: “”}, “capabilities”: [“oob”]}}

[2022-09-24 10:45:18] qubes_gui/init: 573
[2022-09-24 10:45:18] qubes_gui/init: 582
[2022-09-24 10:45:18] qubes_gui/init: 584
[2022-09-24 10:45:18] qubes_gui/init[611]: version sent, waiting for xorg conf
[2022-09-24 10:45:18] {“return”: {}}

[2022-09-24 10:45:18] + ‘[’ ‘!’ -e /tmp/qemu.qmp ]
[2022-09-24 10:45:18] + ‘[’ -e /proc/self/fd/4 ]
[2022-09-24 10:45:18] + ‘[’ -e /proc/self/fd/3 ]
[2022-09-24 10:45:18] + true
[2022-09-24 10:45:18] + printf ‘==== Press enter for shell ====\n’
[2022-09-24 10:45:18] ==== Press enter for shell ====
[2022-09-24 10:45:18] + read
[2022-09-24 10:45:18] + vchan-socket-proxy 0 device-model/46/qmp-vchan /tmp/qemu.qmp
[2022-09-24 10:45:18] br0: port 2(vif46.0-emu) entered blocking state
[2022-09-24 10:45:18] br0: port 2(vif46.0-emu) entered disabled state
[2022-09-24 10:45:18] device vif46.0-emu entered promiscuous mode
[2022-09-24 10:45:18] br0: port 2(vif46.0-emu) entered blocking state
[2022-09-24 10:45:18] br0: port 2(vif46.0-emu) entered forwarding state
[2022-09-24 10:45:18] written 110 bytes to vchan
[2022-09-24 10:45:18] written 34 bytes to vchan
[2022-09-24 10:45:18] [00:06.0] xen_pt_realize: Assigning real physical device 71:00.0 to devfn 0x30
[2022-09-24 10:45:18] [00:06.0] xen_pt_register_regions: IO region 0 registered (size=0x00002000 base_addr=0xee400000 type: 0x4)
[2022-09-24 10:45:18] [00:06.0] xen_pt_config_reg_init: Offset 0x000e mismatch! Emulated=0x0080, host=0x0000, syncing to 0x0000.
[2022-09-24 10:45:18] [00:06.0] xen_pt_config_reg_init: Offset 0x0010 mismatch! Emulated=0x0000, host=0xee400004, syncing to 0xee400004.
[2022-09-24 10:45:18] [00:06.0] xen_pt_config_reg_init: Offset 0x0052 mismatch! Emulated=0x0000, host=0x01c3, syncing to 0x0003.
[2022-09-24 10:45:18] [00:06.0] xen_pt_config_reg_init: Offset 0x0072 mismatch! Emulated=0x0000, host=0x0086, syncing to 0x0080.
[2022-09-24 10:45:18] [00:06.0] xen_pt_config_reg_init: Offset 0x00a4 mismatch! Emulated=0x0000, host=0x8fc0, syncing to 0x8fc0.
[2022-09-24 10:45:19] [00:06.0] xen_pt_config_reg_init: Offset 0x00aa mismatch! Emulated=0x0000, host=0x0010, syncing to 0x0010.
[2022-09-24 10:45:19] [00:06.0] xen_pt_config_reg_init: Offset 0x00b2 mismatch! Emulated=0x0000, host=0x1012, syncing to 0x1012.
[2022-09-24 10:45:19] [00:06.0] xen_pt_pci_intx: intx=1
[2022-09-24 10:45:19] [00:06.0] xen_pt_realize: Real physical device 71:00.0 registered successfully

[2022-09-24 10:45:19] processing error - resetting ehci HC
[2022-09-24 10:45:19] random: crng init done
[2022-09-24 10:49:12] pcifront pci-0: Rescanning PCI Frontend Bus 0000:00

from guid.Win10.log:

(Showing only last 1048576 bytes of file)
0x1), after border calc: x=0, y=144, w=9, h=16
shmimage for 0x4a001d2(remote 0x1), x: 0, y: 144, w: 9, h: 16
do_shm_update for 0x4a001d2(remote 0x1), after border calc: x=0, y=144, w=9, h=16
shmimage for 0x4a001d2(remote 0x1), x: 0, y: 144, w: 9, h: 16
do_shm_update for 0x4a001d2(remote 0x1), after border calc: x=0, y=144, w=9, h=16
…
shmimage for 0x4a001d2(remote 0x1), x: 0, y: 144, w: 9, h: 16
do_shm_update for 0x4a001d2(remote 0x1), after border calc: x=0, y=144, w=9, h=16
shmimage for 0x4a001d2(remote 0x1), x: 0, y: 144, w: 9, h: 16
do_shm_update for 0x4a001d2(remote 0x1), after border calc: x=0, y=144, w=9, h=16
shmimage for 0x4a001d2(remote 0x1), x: 0, y: 144, w: 9, h: 16
do_shm_update for 0x4a001d2(remote 0x1), after border calc: x=0, y=144, w=9, h=16
shmimage for 0x4a001d2(remote 0x1), x: 0, y: 144, w: 9, h: 16
do_shm_update for 0x4a001d2(remote 0x1), after border calc: x=0, y=144, w=9, h=16
libvchan_is_eof
release_all_mapped_mfns running
Obtained 8 stack frames.
work area 0 31 3840 1049
Icon size: 128x128
domain dead
Failed to connect to gui-agent

tavalon · September 24, 2022, 5:02pm

I get the exact same boot error message on a newly created Windows 10 VM when I try to start it with USB controllers attached. Booting without the USB controllers attached works fine.

Sorry, no idea what is causing this or how to fix it.

jay · September 26, 2022, 9:59pm

I also had a linux standalone vm stop working after the upgrade to 4.1. Same errors about being unable to find the boot disk:
Booting from Hard Disk... / Boot failed: could not read the boot disk / Booting from Floppy... / Boot failed: could not read the boot disk

The only PCI device it had on it was the USB controller. Removing the USB controller made it boot as expected, and putting it back on caused the error to reappear.

What is going on here?

enmus · September 27, 2022, 2:28pm

qvm-features < VM_Name > gui 0

and/or

qvm-features < VM_Name > gui-emulated 1

doesn’t help?

tavalon · September 27, 2022, 10:49pm

Tried changing those features (all 4 possible combinations). No success - same boot error each time.

kalmutikko · October 12, 2022, 9:30am

I’m seeing the same behavior when creating a new HVM with any ISO file.

slcoleman · October 12, 2022, 3:34pm

| kalmutikko
October 12 |

| - |

I’m seeing the same behavior when creating a new HVM with any ISO file.

I suspected this would be the case.

I wanted to do a test but have not had the time to try this. I wanted to plug a sata bootable SSD drive into my USB controller assigned to that HVM and see if it actually booted from that drive.

My bet is that a recent Xen update changed the Xen hypervisor to preferentially boot from the assigned hardware rather than the Qubes volume normally passed to it. If that is true then Qubes should add some option to define which volume that the HVM should use for booting. Is there a switch for this?

The question would then be how to controll the HVM’s boot device? There must be a Xen parameter for this and a way to specify it directly from Qubes.

If I am wrong about the above then you can stop reading here.

This would also mean that it could be a new capability in Qubes to store AppVM’s in offline storage devices and then boot whichever SSD is plugged into that generic HVM at the time.

If there was a way to encrypt that offline device then this could actually become a new Qubes feature. Many people have repeatedly asked for a way to hide AppVM’s or just keep them off line, and in some cases this might just work for them. But, Is there a way to decrypt the /boot on the device prior to booting from it?

Perhaps one could insert a ubikey and an ssd and the HVM would only boot if both are seen together? Opal 2.0 SSD’s can install a very small binary that runs to collect a password needed to unlock the device prior to booting, and this feature could be used to facilitate this kind of decrypt/bootstrap sequence in a generic HVM. This would allow for some level of plausible deniability given that the drive is both encrypted and not bootable without a 2FA key of some sort.

tavalon · October 12, 2022, 9:16pm

I tried your idea, using bootable USB sticks rather than an SSD, but unfortunately keep getting the same ‘no bootable device’ error.

Setup: Windows 10 HVM with 2 USB controllers attached as PCI devices. Bootable USB sticks (one Linux, one Qubes) physically plugged in before starting the VM. Tried both available physical USB ports. Same boot error in each case.

Your thoughts about a Xen update changing boot order sounds very plausible to me. Do you think it’s worth submitting this as an official bug? It seems several people have this or a similar issue, and I’ve seen no solution. And perhaps if a Xen master looks at it, the problem becomes trivial.

tavalon · December 28, 2022, 6:46pm

I have found a solution for my problem: lowering the VMs RAM. Maybe it works for you too.

balko · August 27, 2023, 6:56pm

I have absolutely the same bug with USB controller. Windows 10 does not boot with the same messages. It looks like seabios fails to find bootable media of installed Windows 10.

But workaround with reducing memory below 3500 MB does not help me. I reduced it up to 1800 MB and still have the same error: “No bootable device.”.

Any other ideas?

balko · August 27, 2023, 6:58pm

I also tried to connect PCI device to a running Win10 and got an error in journalctl:
dom0 libvirtd[***]: internal error: libxenlight failed to attach pci device
But connecting PCI device in situ is probably not expected to work anyway.