Minimal Template installation fails in Qubes 4.1rc1

ok so we have 4 people confirmed.

try reinstall 4.1rc1 and use fedora-34 as default template, then use sys-firewall to install template.

Updated assumption :

  • debian-11 as default template in initial config; will lead qvm-template error, qubes-dom0-update work.
  • fedora-34 as default template in intial config; qubes-dom0-update is work, qvm-template half work (sys-firewall work, sys-whonix not work)

If anyone wanna test it, please do it with 4.1rc1, I can confirm and remember that with 4.1 beta i don’t have these problem.

Installed Qubes OS 4.1rc1, all default installation settings, fully up to date dom0 and TempateVMs.
Findings:
fedora-34 and fedora-33 based sys-firewall's work with qvm-template but whonix-gw-16, debian-11 and debian-10 based UpdateVM’s for dom0 do not.

debian-11 and whonix-gw-16 based UpdateVMs fail with

Downloading 'qubes-template-debian-11-0:4.0.6-202110081812'...
qubes-template-debian-11-0:4.0.6-202110081812:   0%|                                                                                                                                                                               | 0.00/965M [00:00<?, ?B/s]
'qubes-template-debian-11-0:4.0.6-202110081812' download failed, retrying...
qubes-template-debian-11-0:4.0.6-202110081812:   0%|                                                                                                                                                                               | 0.00/965M [00:00<?, ?B/s]
'qubes-template-debian-11-0:4.0.6-202110081812' download failed, retrying...
qubes-template-debian-11-0:4.0.6-202110081812:   0%|                                                                                                                                                                               | 0.00/965M [00:00<?, ?B/s]
'qubes-template-debian-11-0:4.0.6-202110081812' download failed, retrying...
qubes-template-debian-11-0:4.0.6-202110081812:   0%|                                                                                                                                                                               | 0.00/965M [00:00<?, ?B/s]
'qubes-template-debian-11-0:4.0.6-202110081812' download failed, retrying...
qubes-template-debian-11-0:4.0.6-202110081812:   0%|                                                                                                                                                                               | 0.00/965M [00:00<?, ?B/s]
Error: 'qubes-template-debian-11-0:4.0.6-202110081812' download failed.

debian-10 based UpdateVM fails with

[Qrexec] /usr/lib/qubes/qvm-template-repo-query: line 40: command not found
ERROR: qrexec call 'qubes.TemplateSearch' failed.

fedora-33 based UpdateVM was a bit of a surprise for me as that didn’t work on my main machine (imported from backups fedora-33-minimal based that had all of the required packages installed or at least I thought it had).

Although This issue does not depend on updates via whonix or clearnet but I wanted to ask it here.
Default Template repository location defined in qvm-template --help is /etc/qubes/repo-templates/qubes-templates.repo
Was it same for R4.0 as I have never used 4.0 and onionized repository guide on Whonix site mentions changing /etc/yum.repos.d/qubes-templates.repo

All qvm-template install commands fail with the same error here.

Example:

qvm-template install debian-10

I have created a strace from it, sadly I can’t upload (new user limitation),
so I paste some extract here:

openat(AT_FDCWD, "/home/Echnaton/.cache/qvm-template/tmp9_sa1u3i/qubes-template-debian-10-0:4.0.6-202009131420.rpm.UNTRUSTED", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = 4
fstat(4, {st_mode=S_IFREG|0664, st_size=0, ...}) = 0
ioctl(4, TCGETS, 0x7ffffb93c3e0)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(4, 0, SEEK_CUR)                   = 0
openat(AT_FDCWD, "/etc/qubes/repo-templates/qubes-templates.repo", O_RDONLY|O_CLOEXEC) = 5
fstat(5, {st_mode=S_IFREG|0666, st_size=2006, ...}) = 0
ioctl(5, TCGETS, 0x7ffffb93c270)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(5, 0, SEEK_CUR)                   = 0
ioctl(5, TCGETS, 0x7ffffb93c130)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(5, 0, SEEK_CUR)                   = 0
fstat(5, {st_mode=S_IFREG|0666, st_size=2006, ...}) = 0
read(5, "[qubes-templates-itl]\nname = Qub"..., 2007) = 2006
read(5, "", 1)                          = 0
close(5)                                = 0
socket(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC, 0) = 5
connect(5, {sa_family=AF_UNIX, sun_path="/var/run/qubesd.sock"}, 22) = 0
sendto(5, "admin.vm.Start+ dom0 name sys-fi"..., 39, 0, NULL, 0) = 39
shutdown(5, SHUT_WR)                    = 0
recvfrom(5, "0\0", 8192, 0, NULL, NULL) = 2
recvfrom(5, "", 8192, 0, NULL, NULL)    = 0
close(5)                                = 0
ioctl(2, TCGETS, {B38400 opost isig icanon echo ...}) = 0
pipe2([5, 6], O_CLOEXEC)                = 0
pipe2([7, 8], O_CLOEXEC)                = 0
fstat(6, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
ioctl(6, TCGETS, 0x7ffffb93b760)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(6, 0, SEEK_CUR)                   = -1 ESPIPE (Illegal seek)
fstat(7, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
ioctl(7, TCGETS, 0x7ffffb93b760)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(7, 0, SEEK_CUR)                   = -1 ESPIPE (Illegal seek)
pipe2([9, 10], O_CLOEXEC)               = 0
clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x71ce30d92a10) = 44167
close(10)                               = 0
close(8)                                = 0
close(5)                                = 0
read(9, "", 50000)                      = 0
close(9)                                = 0
write(6, "--releasever=4.1\nqubes-template-"..., 2074) = 2074
close(6)                                = 0
ioctl(2, TIOCGWINSZ, {ws_row=50, ws_col=159, ws_xpixel=0, ws_ypixel=0}) = 0
write(2, "\rqubes-template-debian-10-0:4.0."..., 160) = 160
wait4(44167, 0x7ffffb93c2c4, WNOHANG, NULL) = 0
lseek(4, 0, SEEK_CUR)                   = 0
select(0, NULL, NULL, NULL, {tv_sec=0, tv_usec=100000}) = 0 (Timeout)
wait4(44167, 0x7ffffb93c2c4, WNOHANG, NULL) = 0
lseek(4, 0, SEEK_CUR)                   = 0
write(2, "\rqubes-template-debian-10-0:4.0."..., 160) = 160
select(0, NULL, NULL, NULL, {tv_sec=0, tv_usec=100000}) = ? ERESTARTNOHAND (To be restarted if no handler)
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=44167, si_uid=1000, si_status=127, si_utime=0, si_stime=1} ---
select(0, NULL, NULL, NULL, {tv_sec=0, tv_usec=15046}) = 0 (Timeout)
wait4(44167, [{WIFEXITED(s) && WEXITSTATUS(s) == 127}], WNOHANG, NULL) = 44167
write(2, "\rqubes-template-debian-10-0:4.0."..., 160) = 160
write(2, "\n", 1)                       = 1
close(4)                                = 0
unlink("/home/Echnaton/.cache/qvm-template/tmp9_sa1u3i/qubes-template-debian-10-0:4.0.6-202009131420.rpm.UNTRUSTED") = 0
write(2, "'qubes-template-debian-10-0:4.0."..., 77) = 77
close(7)                                = 0
openat(AT_FDCWD, "/home/Echnaton/.cache/qvm-template/tmp9_sa1u3i/qubes-template-debian-10-0:4.0.6-202009131420.rpm.UNTRUSTED", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = 4
fstat(4, {st_mode=S_IFREG|0664, st_size=0, ...}) = 0
ioctl(4, TCGETS, 0x7ffffb93c3e0)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(4, 0, SEEK_CUR)                   = 0
openat(AT_FDCWD, "/etc/qubes/repo-templates/qubes-templates.repo", O_RDONLY|O_CLOEXEC) = 5
fstat(5, {st_mode=S_IFREG|0666, st_size=2006, ...}) = 0
ioctl(5, TCGETS, 0x7ffffb93c270)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(5, 0, SEEK_CUR)                   = 0
ioctl(5, TCGETS, 0x7ffffb93c130)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(5, 0, SEEK_CUR)                   = 0
fstat(5, {st_mode=S_IFREG|0666, st_size=2006, ...}) = 0
read(5, "[qubes-templates-itl]\nname = Qub"..., 2007) = 2006
read(5, "", 1)                          = 0
close(5)                                = 0
socket(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC, 0) = 5
connect(5, {sa_family=AF_UNIX, sun_path="/var/run/qubesd.sock"}, 22) = 0
sendto(5, "admin.vm.Start+ dom0 name sys-fi"..., 39, 0, NULL, 0) = 39
shutdown(5, SHUT_WR)                    = 0
recvfrom(5, "0\0", 8192, 0, NULL, NULL) = 2
recvfrom(5, "", 8192, 0, NULL, NULL)    = 0
close(5)                                = 0
ioctl(2, TCGETS, {B38400 opost isig icanon echo ...}) = 0
pipe2([5, 6], O_CLOEXEC)                = 0
pipe2([7, 8], O_CLOEXEC)                = 0
fstat(6, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
ioctl(6, TCGETS, 0x7ffffb93b760)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(6, 0, SEEK_CUR)                   = -1 ESPIPE (Illegal seek)
fstat(7, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
ioctl(7, TCGETS, 0x7ffffb93b760)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(7, 0, SEEK_CUR)                   = -1 ESPIPE (Illegal seek)
pipe2([9, 10], O_CLOEXEC)               = 0
clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x71ce30d92a10) = 44168
close(10)                               = 0
close(8)                                = 0
close(5)                                = 0
read(9, "", 50000)                      = 0
close(9)                                = 0
write(6, "--releasever=4.1\nqubes-template-"..., 2074) = 2074
close(6)                                = 0
ioctl(2, TIOCGWINSZ, {ws_row=50, ws_col=159, ws_xpixel=0, ws_ypixel=0}) = 0
write(2, "\rqubes-template-debian-10-0:4.0."..., 160) = 160
wait4(44168, 0x7ffffb93c2c4, WNOHANG, NULL) = 0
lseek(4, 0, SEEK_CUR)                   = 0
select(0, NULL, NULL, NULL, {tv_sec=0, tv_usec=100000}) = 0 (Timeout)
wait4(44168, 0x7ffffb93c2c4, WNOHANG, NULL) = 0
lseek(4, 0, SEEK_CUR)                   = 0
write(2, "\rqubes-template-debian-10-0:4.0."..., 160) = 160
select(0, NULL, NULL, NULL, {tv_sec=0, tv_usec=100000}) = ? ERESTARTNOHAND (To be restarted if no handler)
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=44168, si_uid=1000, si_status=127, si_utime=0, si_stime=0} ---
select(0, NULL, NULL, NULL, {tv_sec=0, tv_usec=22625}) = 0 (Timeout)
wait4(44168, [{WIFEXITED(s) && WEXITSTATUS(s) == 127}], WNOHANG, NULL) = 44168
write(2, "\rqubes-template-debian-10-0:4.0."..., 160) = 160
write(2, "\n", 1)                       = 1
close(4)                                = 0

strace2.log (264.4 KB)

Now upload is possible, so here it is.

It worked for me with Fedora-34 this time.
After installing debian-11-minimal, I shifted sys-net to template based on debian-11-minimal.
After that I tried downloading Whonix 16 with
sudo qubes-dom0-update --enablerepo=qubes-templates-community qubes-template-whonix-gw-16 qubes-template-whonix-ws-16

It errored out with Specify only one package to reinstall template

But trying to download only gw worked even when sys-net is based on debian-minimal based template.

Edit: My bad, sys-firewall was supposed to be of debian-11-minimal based to test further. Tried with debian minimal based sys-firewall and error was-
Redirecting to qvm-template.......
[Qrexec] /usr/lib/qubes/qvm-template-repo-query: line 40: dnf: command not found ERROR: qrexec call 'qubes-TemplateSearch' failed.
This may be of interest to @unman

I also tried updating fedora-34-VM with same minimal setup with following failure
Updating fedora-34 Error exit status 20, failed to download metadata for repo 'updates': cannot prepare internal mirrorlist: curl error (56): Failure when receiving data from peer for https://mirrors.fedoraproject.org/metalink?repo=updates-released-f34&arch=x86_64 [Recv failure: connection reset by peer]

Better description of the issue

@bungali you are right. Root cause is a bug in DVM-debian 11.
It seems , it is on the roadmap for RC2

Problem with a fresh install (not in-place upgrade), there is no DVM-fedora and you can’t create it.

Workarround:

  1. clone debian-dvm name it fedora-34-dvm
  2. start qube settings for fedora34-dvm and change template from debian-11 to fedora-34
  3. go to main menue → Qubes-tools → Qubes-Global-Settings and change for DOM0 UpdateVM the value from debian-11-dvm to fedora-34-dvm