Failing to start fresh fedora-32 template VM

User Support
1

Hi,

I’ve added fedora-32 via sudo qubes-dom0-update qubes-template-fedora-32 in dom0.
When it finished downloading it attemts to start it, but I get:

Domain fedora-32 has failed to start: Cannot connect to qrexec agent for 60 seconds, see /var/log/xen/console/guest-fedora-32.log for details

See the log here: guest-fedora-32.log (281.6 KB)

Now, I’ve tried removing the template and reinstalling a bunch of times:

  • via sudo qubes-dom0-update --action=reinstall qubes-template-fedora-32
  • via sudo dnf remove qubes-template-fedora-32 then sudo qubes-dom0-update qubes-template-fedora-32
  • via qvm-prefs template-vm-name installed_by_rpm false followed by qvm-remove fedora-32 then sudo qubes-dom0-update qubes-template-fedora-32

The result was the same.

I can start fedora-32 qube via Qubes Manager, but it just shows up as constantly consuming 50% CPU and minimal RAM (~400MB). It seems it’s “stuck” in some endless loop. The only way to stop it is to attempt shutdown, followed by “kill qube”.

Edit: to clarify, I do get the “Starting…” and “Started…” notifications. Also, the 50% CPU is what I see when I expand the system tray “Q” icon. When I run System Tools → Task Managed it reports only 1-2% CPU usage. However there must be some serious work done, as I hear my fans spinning like crazy, and as soon as I kill the qube, they settle.

That also means I can’t create any AppVM based on this qube. It will also be stuck at startup with 50% CPU consumed.

What is puzzling to me is that I had this template installed and working on Friday - I was trying to use qubes-builder to create 4.1 ISO, but failed (for other reasons). And this issue only started when I decided to restart the whole process from scratch - removed the AppVM, then TemplateVM.

I do have a full backup of all my qubes and dom0 from before I started all this, but it will take ages, so before I attempt it, I’d like to first try and troubleshoot / identify the issue.

I’m not sure how to progress from here. Could you have a look at the log and help me identify the issue?

2

Weird. Have you tried fedora-33 as well? It’s already available.

Also, do you mind if I reupload the log to your post and remove the pastbin thing? (I’m a moderator).

4

I can’t reupload the log.

Sorry, new users can not upload attachments.

I have not tried fedora-33. I will try now, it’s probably going to take some time to download. I will report back when it finishes.

1 Like
5

The same happened to fedora-33.

After successful installation install-f33.log (956 Bytes) I see “Starting fedora-33” notification followed by:

Domain fedora-33 has failed to start: Cannot connect to qrexec agent for 60 seconds, see /var/log/xen/console/guest-fedora-33.log for details

Then the domain is halted.

guest-fedora-33.log (36.5 KB)

This is the first time I attempted to install and run Fedora 33.

My dom0 is up-to-date and I have not been doing anything fancy in there.

6

Can you try increasing the initial memory from 400MB to 1000MB?

7

I’ve tried 1000MB / fedora-32. Seems to be the same.

Tried qvm-run -q -a fedora-32 xterm to test - nothing.

8

Also tried with 4000 MB / fedora-32 / 4 VCPU. Same result. The only diffrence now Qubes Domains tray sais this qube is using 25% CPU (vs 50% when I started with default 2 VCPUs). So in any scenario, it completely occupies one thread.

9

Interesting, but not necessarily important detail is: Qubes Domains tray always reports 1 MB less than what I set as the initial memory. Eg. when I set 400, the tray reported 399 MB, when I set 1000MB, it reported 999, etc.

1 Like
10

Also, running qvm-features fedora-32 yields only qubesmanager.maxmem_value 4000, while for comparison my fedora-30 base template returns much more:

supported-service.qubes-network 1
supported-service.network-manager 1
qrexec 1
supported-service.clocksync 1
supported-service.modem-manager 1
supported-service.meminfo-writer 1
gui 1
supported-service.qubes-updates-proxy 1
supported-service.qubes-firewall 1
supported-service.qubes-update-check 1
supported-service.updates-proxy-setup 1
qubes-firewall 1
os Linux
supported-service.cups 1
updates-available

It seems to me that the fedora-32 template VM is not configured correctly, probably as a result of that first Start freezing.

11

Eg. when I set 400, the tray reported 399 MB, when I set 1000MB, it reported 999, etc.

Interesting.Try below command in dom0. maybe qmemman is dead. Do you see any error messages in /var/log/qubes/qmemman.log?

sudo systemctl restart qubes-qmemman
12

Done sudo systemctl restart qubes-qmemman, retried starting the qube. Still the same. Looked at /var/log/qubes/qmemman.log and searched for pattern err - nothing found.

Also, my dom0 is takeing 4070 RAM, it seems suspiciously high, though I’m not sure what it was before, but bet it was much lower for the last year.

13

Hmm I thought this problem was similar to this.

14

I can start and use the Template/App VMs based on fedora-30, though it seems they always take whatever is “Max memory”, eg. the VM I’m currently writing from is set between 1200-6000 RAM, is running only firefox with literally 5 tabs, however takes 5983MB. An hour ago it was set to max 12000MB and it did take > 11k (under similar light load).

My point being here, it seems like there is something wrong with memory mangement.

I did try to build qubes ISO recently, for that I used the fedora-32 app vm (so the template did work a few days ago), I did also untick “include in memory management” for that AppVM as it was suggested in docs that it could speed things up.

However after the build failed (for other reason) I did remove that AppVM and the template VM, as I wanted to start the whole process from scratch. Perhaps that untick messed somehow with my system?

EDIT: just found this quote from andrewdavidwong on Reddit

Memory is automatically and dynamically redistributed across VMs (including dom0) as needed by a daemon called qmemman (Qubes memory manager). When you start more VMs, memory will be taken from dom0 (up to a point, of course) and given to those VMs.

In other words, dom0 isn’t actually using that much RAM. It’s just that your system as a whole isn’t using much RAM, so the unused excess is being distributed to whichever VMs happen to be running, and dom0 is always running whenever Qubes is running.

TLDR; all of my memory observations are perfectly normal

15

I’ve cloned a working fedora-30 template VM and will attempt to manually upgrade it to fedora-31, then 32, see if that works.

Update:
It worked - updateing an existing template vm to fedora 32. I was able to build 4.1 ISO and install on my Zenbook (so the Xen patch worked for me).

I’m still confused what went wrong with (official) templates installed “by the book”, but it seems I won’t need them (for now). The fedora-33 template included in 4.1 ISO works without problems.

Thanks for help @deeplow @kommuni . I think this thread can be closed now.

2 Likes