After crash cannot start any VM qubes

User Support 4.1
#1

seems to be same as reported here: Qubesd service not running on boot, cannot be started

boot is slow, but I can eventually login and open dom0 terminal

load is high, but no other qubes start

qvm-ls returns

File "/usr/lib/python3.8/site-packages/qubesadmin/app.py", line 727, in qubesd_call
   client_socket.connect(qubesadmin.config.QUBESD_SOCKET)
FileNotFoundError: [Errno 2] No such file or directory

cannot start Qube Manager

This happened at least once before, but a reboot fixed it. I’ve rebooted about 6 times so far and no luck this time. Crashing has been regular since install 4.1, and more regular since I reinstalled 4.1 on BTRFS to address backup issues. System load is very high on BTRFS and is high now even though no VM qubes are loading.

systemctl status qubesd

gives

 Loaded: loaded
 Active:deactivating

then

systemctl stop qubesd

changes to

Active: failed

then

 systemctl start qubesd

seems to just hang

#2

if i just run
dom0$] qubesd

I get a bunch of output with a final error of

“Permission denied: ‘/var/lib/qubes/appvms/qube-name/private.img’”

for each qube.

Permissions on these .img files are: -rw------- 1 root qubes

inspired by this: Qubes daemon often doesn't start on boot · Issue #5295 · QubesOS/qubes-issues · GitHub

EDIT: Obviously the solution to the permissions error is to run

dom0$]sudo qubesd

then it goes through and “Reflinked” the private.img of each qube to longer named version of each with the date in it

Then it “Renames” a bunch of .imgs
Then it “Removes” a bunch of .imgs
Then it starts "Reflink"ing again

 sudo systemctl status qubesdb

returns

 qubesd.service:start operation timed out. Terminating.
#3

After trying to manually backup the private.img and root.img for some important qubes, I tried rebooting one more time. This time some basic qubes booted. Seems running qubesd above fixed something. This allowed me to run Qubes Backup on the important qubes before reinstalling on XFS.

XFS on RAID1 looks successful(unlike EXT4, which doesn’t work on 4.1 installer) and load so far seems back to normal (unlike BTRFS, which constantly froze). Hopefully found my new file system.

1 Like