Reversing qvm.usb-keyboard

User Support
1

NOTE: If you never installed a sys-usb because qubes setup grayed it out (as you are using a USB keyboard) then I would love for you to reply to this answering the questions in the bottom numbered list. Otherwise I will have to reinstall qubes on another drive in the near future and look closely at that. Thanks.

I followed some instructions at USB qubes | Qubes OS specifically

sudo qubesctl state.sls qvm.usb-keyboard

Which caused troubles. I just want to 100% reverse what I did. But I don’t quite know how.

The trouble started when I rebooted afterwards and got this:

Warning: /dev/mapper/qubes_dom0-root does not exist
Warning: /dev/qubes_dom0/root does not exist
Warning: /dev/qubes_dom0/swap does not exist

I have qubes installed and booting from a USB SSD device. So I thought maybe it is locking itself out of finding the partitions it needs.

I also discovered that I have 2 different USB controllers. Not sure how relevant that is.

BTW, I have a PS/2 keyboard plugged in just in case, but I haven’t needed it.

So I got back in by removing usbcore.authorized_default=0 and adding qubes.skip_autostart to the kernel command line. Thank god for that.

This seems serious enough and easy enough to happen that there should be some warning on this page I think. It points you to “How to remove a USB qube” which is incomplete… hence this post.

I re-ran the salt thing to see all the steps it takes, so I can reverse each one of them.

To reverse this I might need to do these, but maybe not, I cannot tell now:

  1. edit /etc/default/grub and remove the part where it adds “usbcore.authorized_default=0”
  2. If I have to do 1, then I have to do the grub2-mkconfig thing
  3. Remove the sys-usb qube
  4. Edit /etc/qubes-rpc/policy/qubes.InputKeyboard … but to what? Having ‘sys-usb’ lines without a ‘sys-usb’ qube seems wrong to me.
  5. Edit /etc/qubes-rpc/policy/qubes.InputMouse … but to what?
  6. Get rid of /etc/systemd/system/qubes-vm@sys-usb.service.d/50_autostart.conf (manually? systemd disable?)
2

You can leave the USB controller to which your USB drive is connected in dom0 and use another USB controller for sys-usb. You’ll just have to remove the usbcore.authorized_default=0.

On how to revert the sys-usb:

  1. Yes.
  2. Yes.
  3. Yes.
  4. Remove all the lines with sys-usb.
  5. Remove all the lines with sys-usb.
  6. I think you can just remove the whole directory /etc/systemd/system/qubes-vm@sys-usb.service.d.

You can check what sys-usb salt do in these files:

/var/cache/salt/minion/files/base/qvm/usb-keyboard.sls
/var/cache/salt/minion/files/base/qvm/sys-usb.sls
/var/cache/salt/minion/files/base/qvm/hide-usb-from-dom0.sls

Also, I think instead of /etc/qubes-rpc/policy/qubes.InputKeyboard and /etc/qubes-rpc/policy/qubes.InputMouse old policy files the new one is used /etc/qubes/policy.d/50-config-input.policy and you need to change it.

1 Like
3

I’d love to try this, but now that I’ve tried to install QubesOS to a different disk, cancelled midway through, and tried to reboot into the original install… the original install’s boot files have vanished. This is apparently a known thing.

I’d sure love to know what those files were, in particular everything under EFI/ disappeared, and qubes/xen.cfg is missing too.

4

You can install the grub again. Boot from some LiveUSB (e.g. from Qubes OS installer in Rescue) and chroot your Qubes, assuming /dev/sdb is your Qubes drive:
Read how to mount your root:
How to mount a Qubes partition from another OS | Qubes OS
Mount your system:

mkdir /mnt/system
mount /dev/qubes_dom0/root /mnt/system
cd /mnt/system
mount /dev/sdb2 boot
mount /dev/sdb1 boot/efi
mount -t proc /proc proc
mount -t sysfs /sys sys
mount --rbind /dev dev
mount --rbind /run run
mount --rbind /sys/firmware/efi/efivars sys/firmware/efi/efivars
chroot .

If your ESP was deleted and you had to recreate it, then you will have to update it’s UUID in /etc/fstab. Use blkid to list the UUIDs of your devices.
Install grub and add EFI entry;

grub2-install --efi-directory=/boot/efi
efibootmgr -v -c -u -L QubesOS -l /EFI/qubes/grubx64.efi -d /dev/sdb -p 1
1 Like
5

Another great reply with lots of relevant help. Thanks.

I tried this on an ArchLinux system and got really weird LVM issues. So I tried again on qubes rescue and those issues didn’t happen.

I did have to cryptsetup luksOpen DEVICE3 qubes_luks first.

The grub2-install didn’t work, surprisingly the rescue doesn’t have the grub2-efi-x64-modules package, and it gave me this:

grub2-install: error: /usr/lib/grub/x86_64-efi/modinfo.sh doesn't exist. Please specify --target or --directory.

I did this for good measure

grub2-mkconfig -o /boot/efi/EFI/qubes/grub.cfg

and also the efibootmgr command which worked.

And I was then able to successfully reboot into the system, get through grub, unlock LUKS, and then it ran into this lovely little error:

EXT4-fs error (device dm-4): ext4_journal_check_start:83: comm systemd-journal: Detected aborted journal
EXT4-fs (dm-4): Remounting filesystem read-only

At that point the system was frozen. No keyboard response. No disk light activity. I waited a while, then hard powered off.

Redid rescue, this time

# fsck /dev/sdc2
/dev/sdc2: recovering journal
/dev/sdc2: clean, 42/67680 files, 48870/270331 blocks

# fsck /dev/sdc1
0x41: Dirty bit is set. Fs was not properly unmounted and some data may be corrupt.
1) Remove dirty bit
2) No action
? 1
Perform changes ? (y/n) y
/dev/sdc1: 11 files, 26292/147161 clusters

/dev/sdc3 was fine.

Shut down rescue.

Now it didn’t come up in the firmware as a bootable USB anymore.

I suppose I should I rescue again and rerun the efibootmgr command from the chroot, and then check all the LVM partitions to fix the ext4 error wherever that is (among 54 logical volumes!). Too late to do that tonight though.

Thanks for the help so far and I’ll keep you updated

[BTW, I could just rebuild from backups. I would have only lost part of a day’s work and I remember what I did that day… I’d just rather get the experience of recovering this thing, especially for that day when I really do have something I don’t want to lose]

6

To get rid of this problem you can use default EFI boot path. Mount your Qubes EFI partition from another system and copy files:

mkdir /mnt/efi
mount /dev/sdc1 /mnt/efi
cp -r /mnt/efi/EFI/qubes/* /mnt/efi/EFI/BOOT/.
mv /mnt/efi/EFI/BOOT/grubx64.efi /mnt/efi/EFI/BOOT/BOOTX64.efi
mv /mnt/efi/EFI/BOOT/grub.cfg /mnt/efi/EFI/BOOT/BOOTX64.cfg
umount /mnt/efi

GRUB - ArchWiki

7

I’m going to reinstall because I don’t know how to fix dm-4 which is the pool_tdata logical volume for the vm pool, not an ext4 filesystem I can fsck (or delete and re-add the journal from). I don’t understand why ext4_journal_check_start was trying to check that logical volume… or maybe the ‘dm’ numbers aren’t the same across reboots. In any case I did fsck on all the ext4 file systems and it didn’t fix this.

I’m going to buy a replacement SSD so I don’t have to overwrite this one. I may just copy files off of this one.

8

I am also trying to reinstall grub as I accidentally wrote over my qubes grub after installing debian on a seperate drive, forgetting to change the bootloader drive for debian to my debian disk and not my qubes disk.

When starting Qubes from the rescue disk, it automatically mounts my root Qubes partition with
cryptsetup open /dev/sda luks-qubes after asking for my root partition password.

This prevents me from proceeding with

cd /mnt/system
mount /dev/sda2 boot

Giving me the error that the LUKS partition is already mounted on luks-qubes.
I tried to unmount luks-qubes, but can’t because it is in use from the following command:
mount /dev/qubes_dom0/root /mnt/system.

These instructions are not clear, and already on step 3 I am encountering problems. How can I mount /dev/sda2 to /mnt/system/boot when /dev/sda2 is already mounted on luks-qubes? I can’t unmount luks-qubes either because dom0 is using it on /mnt/system/.

How can I proceed from here?

9

Aren’t you supposed to mount sda1 for your boot partition? sda2 can be either swap or the luks partition with your data (so it’s normal that it tells you it’s already mounted).