[Help data recovery]All VMs except dom0 showing size 0

I have a 512 GB internal SSD that is my main machine running only qubes. Since, I was new to qubes I never setup backup and procrastinated backup.

On fine day while watching youtube system hung and I hard resettled my laptop. After boot no new VMs started. So, I took it out and plugged SSD externally to another computer and booted same behavior.

I needed 2 help.

  1. How do I backup an image of the SSD on to external HDD(1TB), verify the image by booting with it.
  2. Iteratively try to restore the Data in SSD. Iteratively meaning if I screwup I want to load data from HHD to SSD and try again.

Some command line output:

  1. qubes manager backup menu showing sizes:
  2. lvm scan list and details:


lvchange -ay

change anything?

If you have any valuable information on this SSD - then the first thing you should do is to backup it now by making an image.
If you have a lot of time and a spare storage of the SSD size - you can make a full copy using dd from SSD to a file on some ext4/NTFS or other partition and hashsum it after that (Note: not FAT32 due to 4GiB limit).
Do not do this backup from the Qubes OS and better do not boot it at all before making backup (in case it is shut down already). Just boot using Kubuntu LiveCD, mount external HDD, and make a copy with dd to a file (need help with that?).
Alternatively you can make a copy with clonezilla, but it can make a modified image of SSD (partition table is taking into account, what’s probably not what you want, compression can be added).

Having the image of SSD in a file on external disk you can disconnect SSD and temporary put it away. The next step is to mount created image with losetup (in a read-only mode) and try to extract all desired data from the LVM pool. Qubes OS Docs had an article about manual mounting and extracting data from pools.

Hi @balko I have valuable data in some VMs. I have a spare laptop running ubuntu and a external HDD 1TB(work on windows, so not sure if it is FAT32, need to check)
Process you mentioned is what I want to do and get the data out

Can you comment if I can create the backup image lossless, such that so that I can load it back to internal SSD and try recovering LVMs, so that I can avoid fresh Qubes install is possible (I may have to do this multiple times)

sudo dd if=/dev/YOURSSD of=full_image.bin conv=fsync status=progress
will make “lossless” copy of your SSD (something like /dev/sda, check name in partitionmanager or fdisk) to full_image.bin that will be located to the current directory. If you have Windows installed on external HDD it will have NTFS almost certainly, that is OK filesystem for that task as GNU/Linux supports it well enough.

But I did not understand the part about uploading back to SSD, you mean another one of the same size or what? If the data on the SSD is corrupted in any way (does not matter if it is software/LVM or physical issue) I would not recommend to have a system in this state even on a new SSD.

I understood the first part, I might be able to test the full_image.bin by booting using QEMU or virtual machine manager, right?

The second part, I want to do reverse process to load to the same SSD, you can think of it as dd if=full_image.bin of=/dev/YOURSSD

Then boot from SSD to dom0, try some restoration instruction I found on web, including above comment by @ChrisA. If these commands screw up the SSD data more and make it unbootable. Then I will go back and do dd if=full_image.bin of=/dev/YOURSSD few more time to try and restore the LVM tables.

I do not think my SSD is physically damaged, since it is booting into dom0 now. I think the problem is with Qubes VMs tables, so if I can fix it, I may be restore the Qubes to old state.

I think - yes. After making a full snapshot you will be able to try indefinitely. Just keep in on reliable device(s) and checksum it after creation. If you have a lot of time and space you can make both dd version and clonezilla one.

This I do not know. Running Qubes OS inside any VM is not easy. But you definitely can mount the file and analyze data of all VMs inside LVM, all user data may be possible to extract this way (depend on what are the issues with LVM if any).

@balko , Thanks for all the inputs, let me give your instructions a try. I will let you know if I have more questions.