VMs fail to start periodically - libxenlight error

User Support
#1

I have been using Qubes for about a year and I keep having the same problem sporadically. It has not been something permanent so I sort of lived with it, just rebooting when necessary. I just upgraded to 4.1 to see if that solved the problem and I am getting the same errors again.

Basically after I shutdown a VM, I have trouble restarting the qube getting the Libxenlight failed to create domain ‘X’ error which others have seen. It is not one particular qube. It happens randomly with most of them…maybe 1/3 to 1/4 times I try to start up a qube. And it is not permanent. Usually if I restart the computer and everything is reset I can start it back up again. But if I have not encountered the error yet and restart with that qube, the error manifests. For example it is not uncommon if I shutdown the computer and then restart it, randomly one or more qubes give me the error. If I shutdown again, the error goes away. It does not matter if the qube is shutdown gracefully, killed, gets shutdown by the OS while the OS is shutting down, or the power goes out and the entire thing goes down. I have had issues on all these types of shutdowns.

I saw that sometimes people got rid of the network connection or tried to just connect to sys-net and that resolved the issue. Well that does not work in this case. I just tried in. In other cases it appears to be a more permanent situation than what is going on here. Rebooting once encountered literally solves the problem most of the times. And it is not all qubes, just randomly one or more qubes with most of the rest working fine. Happens in 4.0 and 4.1.

Any suggestions as to what might be happening?

#2

Any suggestions from anyone?

#3

When that happens, can you take a look into /var/log/libvirt/libxl/libxl-driver.log in dom0? It should give some more details what exactly has failed.

#4

Thanks for your response.

I got the following errors
xc_ panic: xc_dom_boot.c:120: xc_dom_boot_mem_init: can’t allocate low memory for domain: Out of memory
libxl: libxl_dom.c:593:libxl__build_dom: xc_dom_boot_mem_init failed: Operation not permitted
libxl: libxl_create.c:1623:domcreate_rebuild_done: Domain 49:cannot (re-_build domain: -3

I had the low memory at 12000 MB, and then reduced to 8000 MB and still did not boot, but when i put at 800 MB it launched. So that seems to be the root of the problem.

However, what I do not understand is how I am running into memory issues when I have a full 128 Gigabytes of memery. That is less than 10%. And this was one of the first VMs I tried to launch when I rebooted the computer. And then after it started I was able to launch a bunch of VMs.

Is there some tool that allows one to see how memory is being allocated between dom0 and the other qubes to figure out where what I thought was more memory than I could ever use is disappearing to?

#5

I’m having the exact same problem. The /var/log/libvirt/libxl/libxl-driver.log file in dom0 has the same output as in the following thread:

xc_dom_boot_mem_init: can't allocate low memory for domain: Out of memory

libxl-driver.log (2.9 KB)

I tried restarting libvirtd ( systemctl restart libvirtd ) as suggested but the error keeps occurring.

I assume that libvirt is not managing to reallocate the memory from running qubes to start the new one. I’m using few applications (firefox w/ 3 tabs, thunderbird and audio player) in only two qubes and have 32 GB RAM, so it’d seem that memory is not a problem. In my case, stopping the qube with the audio player allowed me to start the new qube (its minimum memory is at 1700 MB).

Is there some tool that allows one to see how memory is being allocated between dom0 and the other qubes to figure out where what I thought was more memory than I could ever use is disappearing to?

I usually click on the Qubes Manager icon in the sys-tray and the memory used by each qube is shown. However as I understand, the memory allocated to each qube is not exactly the total memory used by the processes – the available memory is distributed among qubes according to their needs. But I don’t know how to check the actual memory being used for the processes to calculate total free memory.