Qvm-start HVM failing, NixOS

User Support
1

When I attempt to start my NixOS HVM it fails, but only sometimes.

I receive a notification that says Start failed: internal error: libxenlight failed to create new domain 'NixOS-00', see /var/log/libvirt/libxl/libxl-driver.log for details

2025-02-27 05:47:05.068+0000: libxl: libxl_pci.c:1587:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:00:0d.0
2025-02-27 05:47:05.079+0000: libxl: libxl_pci.c:1587:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:00:0d.2
2025-02-27 05:47:05.083+0000: libxl: libxl_pci.c:1587:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:00:14.0
2025-02-27 09:11:04.818+0000: libxl: libxl_pci.c:1587:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:00:0d.0
2025-02-27 09:11:04.933+0000: libxl: libxl_pci.c:1587:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:00:0d.2
2025-02-27 09:11:05.047+0000: libxl: libxl_pci.c:1587:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:00:14.0
2025-02-27 09:11:37.450+0000: libxl: libxl_pci.c:1587:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:00:0d.0
2025-02-27 09:11:37.460+0000: libxl: libxl_pci.c:1587:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:00:0d.2
2025-02-27 09:11:37.463+0000: libxl: libxl_pci.c:1587:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:00:14.0
2025-02-27 09:14:02.123+0000: xc: panic: xg_dom_x86.c:1319: meminit_pv: failed to allocate 0x9000 pages: Internal error
2025-02-27 09:14:02.123+0000: xc: panic: xg_dom_boot.c:119: xc_dom_boot_mem_init: can't allocate low memory for domain: Out of memory
2025-02-27 09:14:02.123+0000: libxl: libxl_dom.c:586:libxl__build_dom: xc_dom_boot_mem_init failed: Device or resource busy
2025-02-27 09:14:02.124+0000: libxl: libxl_dm.c:2808:stubdom_pvqemu_cb: Domain 29:error connecting nics devices: Device or resource busy
2025-02-27 09:14:02.124+0000: libxl: libxl_create.c:1975:domcreate_devmodel_started: Domain 29:device model did not start: -3
2025-02-27 09:14:14.751+0000: libxl: libxl_domain.c:81:libxl__domain_rename: Domain 32:Domain with name "NixOS-00-dm" already exists.
2025-02-27 09:14:14.751+0000: libxl: libxl_dm.c:2808:stubdom_pvqemu_cb: Domain 31:error connecting nics devices: Function not implemented
2025-02-27 09:14:14.751+0000: libxl: libxl_create.c:1975:domcreate_devmodel_started: Domain 31:device model did not start: -6
2025-02-27 09:14:32.311+0000: libxl: libxl_domain.c:81:libxl__domain_rename: Domain 34:Domain with name "NixOS-00-dm" already exists.
2025-02-27 09:14:32.311+0000: libxl: libxl_dm.c:2808:stubdom_pvqemu_cb: Domain 33:error connecting nics devices: Function not implemented
2025-02-27 09:14:32.311+0000: libxl: libxl_create.c:1975:domcreate_devmodel_started: Domain 33:device model did not start: -6
2025-02-27 15:16:27.688+0000: libxl: libxl_domain.c:81:libxl__domain_rename: Domain 50:Domain with name "NixOS-00-dm" already exists.
2025-02-27 15:16:27.688+0000: libxl: libxl_dm.c:2808:stubdom_pvqemu_cb: Domain 49:error connecting nics devices: Function not implemented
2025-02-27 15:16:27.688+0000: libxl: libxl_create.c:1975:domcreate_devmodel_started: Domain 49:device model did not start: -6
2025-02-27 15:50:56.636+0000: libxl: libxl_domain.c:81:libxl__domain_rename: Domain 52:Domain with name "NixOS-00-dm" already exists.
2025-02-27 15:50:56.637+0000: libxl: libxl_dm.c:2808:stubdom_pvqemu_cb: Domain 51:error connecting nics devices: Function not implemented
2025-02-27 15:50:56.637+0000: libxl: libxl_create.c:1975:domcreate_devmodel_started: Domain 51:device model did not start: -6
2025-02-27 15:51:43.619+0000: libxl: libxl_domain.c:81:libxl__domain_rename: Domain 54:Domain with name "NixOS-00-dm" already exists.
2025-02-27 15:51:43.620+0000: libxl: libxl_dm.c:2808:stubdom_pvqemu_cb: Domain 53:error connecting nics devices: Function not implemented
2025-02-27 15:51:43.620+0000: libxl: libxl_create.c:1975:domcreate_devmodel_started: Domain 53:device model did not start: -6
2025-02-27 15:51:57.772+0000: libxl: libxl_domain.c:81:libxl__domain_rename: Domain 56:Domain with name "NixOS-00-dm" already exists.
2025-02-27 15:51:57.772+0000: libxl: libxl_dm.c:2808:stubdom_pvqemu_cb: Domain 55:error connecting nics devices: Function not implemented
2025-02-27 15:51:57.773+0000: libxl: libxl_create.c:1975:domcreate_devmodel_started: Domain 55:device model did not start: -6
2025-02-27 16:01:10.080+0000: libxl: libxl_domain.c:81:libxl__domain_rename: Domain 58:Domain with name "NixOS-00-dm" already exists.
2025-02-27 16:01:10.081+0000: libxl: libxl_dm.c:2808:stubdom_pvqemu_cb: Domain 57:error connecting nics devices: Function not implemented
2025-02-27 16:01:10.081+0000: libxl: libxl_create.c:1975:domcreate_devmodel_started: Domain 57:device model did not start: -6

Above are the logs across the full day and attempting start several times.
I’m not out of memory. Giving it less memory or turning off memory sharing has no effect. I suspect something has not fully shutdown since this never happens on first qvm-start.

Any advice?

2

Do you see anything at all?

What are your settings?

Do you have PCI devices attached to the guest?

Please run it in Debug Mode and check.

Can you upload an HCL please?

3

This does say that it appears you don’t have enough RAM to run it?

2025-02-27 09:14:02.123+0000: xc: panic: xg_dom_boot.c:119: xc_dom_boot_mem_init: can’t allocate low memory for domain: Out of memory

4

I have more than enough ram. i tried lowering it by over 30 gbs and it still fails. I don’t think it’s the problem and that error doesn’t show back up when it is raised.

No window shows. There are no devices connected.

prefs

audiovm               D  dom0
autostart             -  False
backup_timestamp      U
debug                 D  False
default_dispvm        D  default-dvm
default_user          D  user
dns                   D  10.139.1.1 10.139.1.2
gateway               D  
gateway6              D  
guivm                 D  dom0
icon                  D  standalonevm-blue
include_in_backups    D  True
installed_by_rpm      D  False
ip                    D  10.137.0.80
ip6                   D  
kernel                -  
kernelopts            D  
keyboard_layout       D  us++
klass                 D  StandaloneVM
label                 -  blue
mac                   D  00:16:3e:5e:6c:00
management_dispvm     D  default-mgmt-dvm
maxmem                -  50000
memory                -  8192
name                  -  NixOS-00
netvm                 D  ndisp-mul-net-02
provides_network      D  False
qid                   -  80
qrexec_timeout        D  60
shutdown_timeout      D  60
start_time            D  
stubdom_mem           U
stubdom_xid           D  -1
template_for_dispvms  D  False
updateable            D  True
uuid                  -  6224cd85-4bd3-46e1-9824-b319687cd719
vcpus                 -  12
virt_mode             -  hvm
visible_gateway       D  10.138.18.120
visible_gateway6      D  
visible_ip            D  10.137.0.80
visible_ip6           D  
visible_netmask       D  255.255.255.255
xid                   D  -1

hcl:

 qubes-hcl-report NixOS-00

Qubes release 4.2.4

Brand:		System76
Model:		Darter Pro
BIOS:		2024-01-18_3e19b73

Xen:		4.17.5
Kernel:		6.6.77-1

RAM:		98147 Mb

CPU:
  13th Gen Intel(R) Core(TM) i5-1340P
Chipset:
  Intel Corporation Raptor Lake-P/U 4p+8e cores Host Bridge/DRAM Controller [8086:a707]
VGA:
  Intel Corporation Raptor Lake-P [Iris Xe Graphics] [8086:a7a0] (rev 04) (prog-if 00 [VGA controller])
  

Net:
  Intel Corporation Raptor Lake PCH CNVi WiFi [8086:51f1] (rev 01)
  Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 15)

SCSI:


HVM:		Active
I/O MMU:	Active
HAP/SLAT:	Yes
TPM:		Device present (TPM 2.0)
Remapping:	yes
Certified:	no
5

wait wtf are those devices in the error log?
they are my usb controllers. Why are they connected at all?
dom0:00_0d.2 USB controller: Intel Corporation Raptor Lake-P Thunderbolt 4 NHI #0
and
dom0:00_14.0 USB controller: Intel Corporation Alder Lake PCH USB 3.2 xHCI Host Controller

what does it meaaaan

2024-12-16 23:18:04.382+0000: libxl: libxl_pci.c:1587:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:00:0d.2
2024-12-16 23:18:04.388+0000: libxl: libxl_pci.c:1587:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:00:14.0
2024-12-16 23:19:16.398+0000: libxl: libxl_pci.c:1582:libxl__device_pci_reset: write to /sys/bus/pci/devices/0000:00:14.3/reset returned -1: Inappropriate ioctl for device
2024-12-16 23:19:17.535+0000: libxl: libxl_qmp.c:1837:qmp_ev_parse_error_messages: Domain 32:Could not open '/sys/bus/pci/devices/0000:00:01.3/config': No such file or directory
2024-12-16 23:19:17.536+0000: libxl: libxl_pci.c:1275:pci_add_qmp_device_add_cb: Domain 32:Retrying PCI add 1
2024-12-16 23:19:18.608+0000: libxl: libxl_qmp.c:1837:qmp_ev_parse_error_messages: Domain 32:Could not open '/sys/bus/pci/devices/0000:00:01.3/config': No such file or directory
2024-12-16 23:19:18.608+0000: libxl: libxl_pci.c:1275:pci_add_qmp_device_add_cb: Domain 32:Retrying PCI add 2
2024-12-16 23:19:19.680+0000: libxl: libxl_qmp.c:1837:qmp_ev_parse_error_messages: Domain 32:Could not open '/sys/bus/pci/devices/0000:00:01.3/config': No such file or directory
2024-12-16 23:19:19.681+0000: libxl: libxl_pci.c:1275:pci_add_qmp_device_add_cb: Domain 32:Retrying PCI add 3
2024-12-16 23:19:20.752+0000: libxl: libxl_qmp.c:1837:qmp_ev_parse_error_messages: Domain 32:Could not open '/sys/bus/pci/devices/0000:00:01.3/config': No such file or directory
2024-12-16 23:19:20.753+0000: libxl: libxl_pci.c:1275:pci_add_qmp_device_add_cb: Domain 32:Retrying PCI add 4
2024-12-16 23:19:21.824+0000: libxl: libxl_qmp.c:1837:qmp_ev_parse_error_messages: Domain 32:Could not open '/sys/bus/pci/devices/0000:00:01.3/config': No such file or directory
2024-12-16 23:19:21.824+0000: libxl: libxl_pci.c:1275:pci_add_qmp_device_add_cb: Domain 32:Retrying PCI add 5
2024-12-16 23:19:22.896+0000: libxl: libxl_qmp.c:1837:qmp_ev_parse_error_messages: Domain 32:Could not open '/sys/bus/pci/devices/0000:00:01.3/config': No such file or directory
2024-12-16 23:19:22.897+0000: libxl: libxl_pci.c:1275:pci_add_qmp_device_add_cb: Domain 32:Retrying PCI add 6
2024-12-16 23:19:23.968+0000: libxl: libxl_qmp.c:1837:qmp_ev_parse_error_messages: Domain 32:Could not open '/sys/bus/pci/devices/0000:00:01.3/config': No such file or directory
2024-12-16 23:19:23.968+0000: libxl: libxl_pci.c:1275:pci_add_qmp_device_add_cb: Domain 32:Retrying PCI add 7
2024-12-16 23:19:25.039+0000: libxl: libxl_qmp.c:1837:qmp_ev_parse_error_messages: Domain 32:Could not open '/sys/bus/pci/devices/0000:00:01.3/config': No such file or directory
2024-12-16 23:19:25.040+0000: libxl: libxl_pci.c:1275:pci_add_qmp_device_add_cb: Domain 32:Retrying PCI add 8
2024-12-16 23:19:26.112+0000: libxl: libxl_qmp.c:1837:qmp_ev_parse_error_messages: Domain 32:Could not open '/sys/bus/pci/devices/0000:00:01.3/config': No such file or directory
2024-12-16 23:19:26.113+0000: libxl: libxl_pci.c:1275:pci_add_qmp_device_add_cb: Domain 32:Retrying PCI add 9
2024-12-16 23:19:27.183+0000: libxl: libxl_qmp.c:1837:qmp_ev_parse_error_messages: Domain 32:Could not open '/sys/bus/pci/devices/0000:00:01.3/config': No such file or directory
2024-12-16 23:19:27.184+0000: libxl: libxl_pci.c:1275:pci_add_qmp_device_add_cb: Domain 32:Retrying PCI add 10
2024-12-16 23:19:27.318+0000: libxl: libxl_pci.c:1864:device_pci_add_done: Domain 32:libxl__device_pci_add failed for PCI device 0:0:14.3 (rc -9)
2024-12-16 23:19:27.319+0000: libxl: libxl_create.c:2000:domcreate_attach_devices: Domain 32:unable to add pci devices
2024-12-16 23:20:38.257+0000: libxl: libxl_pci.c:1582:libxl__device_pci_reset: write to /sys/bus/pci/devices/0000:00:14.3/reset returned -1: Inappropriate ioctl for device
2024-12-16 23:20:48.322+0000: libxl: libxl_device.c:1507:libxl__wait_for_backend: Backend /local/domain/0/backend/pci/33/0 not ready
2024-12-16 23:20:58.334+0000: libxl: libxl_device.c:1218:device_backend_callback: Domain 33:unable to remove device with path /local/domain/0/backend/vbd/33/51712
2024-12-16 23:20:58.339+0000: libxl: libxl_device.c:1218:device_backend_callback: Domain 33:unable to remove device with path /local/domain/0/backend/vbd/33/51728
2024-12-16 23:20:58.341+0000: libxl: libxl_device.c:1218:device_backend_callback: Domain 33:unable to remove device with path /local/domain/0/backend/vbd/33/51744
2024-12-16 23:20:58.345+0000: libxl: libxl_device.c:1218:device_backend_callback: Domain 33:unable to remove device with path /local/domain/0/backend/vbd/33/51760
2024-12-16 23:20:58.348+0000: libxl: libxl_domain.c:1589:devices_destroy_cb: Domain 33:libxl__devices_destroy failed
2024-12-16 23:20:58.368+0000: libxl: libxl_device.c:1218:device_backend_callback: Domain 32:unable to remove device with path /local/domain/0/backend/vbd/32/51712
2024-12-16 23:20:58.370+0000: libxl: libxl_device.c:1218:device_backend_callback: Domain 32:unable to remove device with path /local/domain/0/backend/vbd/32/51728
2024-12-16 23:20:58.371+0000: libxl: libxl_device.c:1218:device_backend_callback: Domain 32:unable to remove device with path /local/domain/0/backend/vbd/32/51744
6

I fixed it by shutting down the -dm domain with xl destroy NixOS-00-dm which I learned about from here: Qubes OS and CPU affinity - #12 by tzwcfq

also i lowered the ram from 50gb to something reasonable as that was too high. Both required to launch.

7

The whole reason I said about RAM was because of this one line…

I am glad that you have found a resolution to your problem.

1 Like
8

yep u were right. the -dm also had to die tho