Loop block device mounts fail after upgrade to v4.2

Has anyone else had issues with mounting loop devices between Qubes recently?

I’ve had this issue intermittently before, but after I did an upgrade to Qubes 4.2, I get the issue every time. I can no longer use loop devices that have been attached from one Qube to another.

Here’s my setup: I have a Qube named source (Debian-12) with some file volume.img that I’m trying to share (as a loop device via qvm-block) with a Qube named destination (I’ve tried both Debian-12 and Fedora-34)

user@source:~$ ls
volume.img
user@source:~$ 

I attach this file volume as a loop device

user@source:~$ sudo losetup -f --show volume.img
/dev/loop0
user@source:~$ 

Then I share the volume either with the GUI or, for clarity in this post, with the qvm-block command in dom0

[root@dom0 ~]# qvm-block
BACKEND:DEVID   DESCRIPTION   USED BY
source:loop0   /home/user/volume.img
[root@dom0 ~]# 

[root@dom0 ~]# qvm-block attach destination source:loop0
[root@dom0 ~]# 

That command exited successfully, and the GUI shows that loop0 is attached from source to destination.

However:

  1. There is no output indicating a new device in dmesg on the destination Qube
  2. There is no new /dev/xvdX device on the destination Qube
  3. The output of ls -lah /dev/disk/* (on the destination Qube) is identical both before and after the qvm-block command has executed above
  4. When I attempt to detach loop0 from destination, I get an error after ~10 seconds
[root@dom0 ~]# time qvm-block detach destination source:loop0
Got empty response from qubesd. See journald in dom0 for details.

real   0m10,082s
user   0m0,033s
sys    0m0,019s

What should happen is that a new disk should pop-up in /dev/disk/by-label/ on the destination Qube, which I can then mount like any other block device. This used to work fine.

Has anyone else experienced this issue with mounting loop devices between Qubes? Any idea how I can debug what’s going on?

Since you upgraded your system from 4.1 to 4.2, are all qubes related packages in those templates based on 4.2, or do you still see some 4.1 packages there?

Since you upgraded your system from 4.1 to 4.2, are all qubes related packages in those templates based on 4.2, or do you still see some 4.1 packages there?

How would I check that?

Qubes package versioning is based on the Qubes release, so if you run dpkg -l | grep -i qubes on debian or rpm -qa | grep -i qubes on fedora, you should be able to check if all packages have been successfully upgraded.

Also, can you give more information about how you created the volume.img volume? Have you tried using the -P option with losetup?

Qubes package versioning is based on the Qubes release, so if you run dpkg -l | grep -i qubes on debian or rpm -qa | grep -i qubes on fedora, you should be able to check if all packages have been successfully upgraded.

Yeah, I see a lot of 4.1.X+deb12u1 output from dpkg -l | grep -i qubes

Is this an issue? How do I fix it?

Also, can you give more information about how you created the volume.img volume? Have you tried using the -P option with losetup?

I used dd and head. I created this volume years ago, and it’s worked fine for years.

It might be. But it’s better to upgrade anyway.
Download the following script and run it inside the template:

Using sudo losetup -fP --show volume.img doesn’t make any difference?

I just tried that, and I get the same problem.

Another thing to try would be to download a fresh debian-12-xfce template from dom0 with qvm-template install debian-12-xfce and try using 2 app qubes based on that template and see if the same error appears.

download a fresh debian-12-xfce template from dom0 with qvm-template install debian-12-xfce

What does that do if I already have a TemplateVM named debian-12

It will create a new one called debian-12-xfce, it won’t change the other one.

This worked.

Well, I didn’t download the script (I’m not a big fan of downloading and executing random scripts from GitHub), but I did see that the qubes-r4.list file still referenced r4.1, so I manually edited it and ran the apt commands.

user@debian-12:~$ sudo su -
root@debian-12:~# cd /etc/apt/sources.list.d/
root@debian-12:/etc/apt/sources.list.d# 

root@debian-12:/etc/apt/sources.list.d# cp qubes-r4.list qubes-r4.list.20241215.bak
root@debian-12:/etc/apt/sources.list.d# 

root@debian-12:/etc/apt/sources.list.d# vim qubes-r4.list
root@debian-12:/etc/apt/sources.list.d# 

root@debian-12:/etc/apt/sources.list.d# diff qubes-r4.list.20241215.bak qubes-r4.list
2c2
< deb [arch=amd64] https://deb.qubes-os.org/r4.1/vm bookworm main
---
> deb [arch=amd64 signed-by=/usr/share/keyrings/qubes-archive-keyring-4.2.gpg] https://deb.qubes-os.org/r4.2/vm bookworm main
root@debian-12:/etc/apt/sources.list.d# 

root@debian-12:/etc/apt/sources.list.d# apt-get clean
root@debian-12:/etc/apt/sources.list.d# 

root@debian-12:/etc/apt/sources.list.d# apt-get update -o Dpkg::Options::="--force-confdef"
Hit:1 http://deb.debian.org/debian bookworm-backports InRelease                
Hit:2 https://deb.debian.org/debian bookworm InRelease                         
Get:3 https://deb.qubes-os.org/r4.2/vm bookworm InRelease [4,739 B]
Hit:4 https://deb.debian.org/debian-security bookworm-security InRelease
Fetched 4,739 B in 2s (3,039 B/s)
Reading package lists... Done
root@debian-12:/etc/apt/sources.list.d# 

root@debian-12:/etc/apt/sources.list.d# apt-get dist-upgrade -y --no-install-recommends -o Dpkg::Options::="--force-confdef"
...
root@debian-12:/etc/apt/sources.list.d# 

I confirmed that my qubes packages in the Debian 12 TemplateVM now say 4.2 instead of 4.1

root@debian-12:/etc/apt/sources.list.d# dpkg -l | grep -i qubes | tail -n2
ii  xserver-xorg-qubes-common                     4.2.19-1+deb12u1                        amd64        Common functions for qubes xserver driver
ii  xserver-xorg-video-dummyqbs                   4.2.19-1+deb12u1                        amd64        Dummy X video driver for qubes-gui-agent
root@debian-12:/etc/apt/sources.list.d# 

Now I’m going to restart and see if that fixes the issue.

The script is made by the Qubes devs, it is used with the qubes-dist-upgrade script that you probably used to upgrade from 4.1 to 4.2.

Update: it worked!

I have no idea what the issue was, but now that I’ve updated the Debian-12 TemplateVM to use the 4.2 repo and its 4.2 packages (instead of 4.1), I’m now able to share loop volumes between VMs.

Thanks so much for your help <3

1 Like

Well I tried the same thing for fedora-34, but I get an error

root@fedora-34 yum.repos.d]# dnf distro-sync -y --best --allowerasing
Qubes OS Repository for VM (updates)             93  B/s | 153  B     00:01    
Errors during downloading metadata for repository 'qubes-vm-r4.2-current':
  - Status code: 404 for https://yum.qubes-os.org/r4.2/current/vm/fc34/repodata/repomd.xml (IP: 127.0.0.1)
Error: Failed to download metadata for repo 'qubes-vm-r4.2-current': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
[root@fedora-34 yum.repos.d]#

I check in some other DispVM, and I can confirm that this URL is fine

But this one is 404

user@disp9071:~$ curl -I https://yum.qubes-os.org/r4.1/current/vm/fc34/
HTTP/1.1 200 OK
Server: nginx/1.22.1
Date: Sun, 15 Dec 2024 19:13:26 GMT
Content-Type: text/html
Connection: keep-alive
Strict-Transport-Security: max-age=63072000; includeSubDomains

user@disp9071:~$ curl -I https://yum.qubes-os.org/r4.2/current/vm/fc34/
HTTP/1.1 404 Not Found
Server: nginx/1.22.1
Date: Sun, 15 Dec 2024 19:13:32 GMT
Content-Type: text/html
Content-Length: 153
Connection: keep-alive
Strict-Transport-Security: max-age=63072000; includeSubDomains

user@disp9071:~$ 

Is there actually an outage right now?

Fedora 34 is very old, you need at least Fedora 37 to upgrade to 4.2 repositories.
You can download a new template or upgrade version by version until you reach 37 or the latest supported 4.1 version (Fedora 39). Then you can upgrade to 4.2 repositories.

Oh wow, thanks. I’m surprised that Qubes never forced me (or even notified me!) to update fedora.

I’m downloading a newer fedora with

[root@dom0 ~]# qvm-template install fedora-41

I also see there’s an xfce version, but I don’t get it – if Qubes’ dom0 is responsible for the actual rendering of the windows (dom0 is what runs X with xfce), then what does it matter if I install fedora-41 vs fedora-41-xfce. What’s the difference?

The in-template applications will be based on xfce instead of gnome. That’s about it. It also means the template will be lighter and run smoother, gnome can have some issues sometimes.

1 Like