Some fedora VMs are strangely broken with template update

I migrated my VMs from legacy fedora-38 to fedora-40 and now SOME of them fail, and I do not know why!

When I try to start any app, nothing happens, in the log file I see obscure message systemctl[XXX]: Failed to connect to bus: no medium found.

Meanwhile, I checked a few things:
systemctl status shows no failed units,
moving either ~/.local or ~/.config away does not help
rexec agent works properly

WTF?!

Are you able to start the GUI terminal in these problematic qubes?
Or are you accessing the console of these qubes using qvm-console-dispvm in dom0?

No, no application works terminal included. Only console and qvm-run.

What if you run some GUI app from the qube’s console e.g. xterm?

I broke some qubes trying to migrate directly from 38 to 40. Then I realized I had to do it 2-step. If that’s what you did, it wasn’t clear from your OP.

What if you run some GUI app from the qube’s console e.g. xterm?

Works as long as I am root.

I broke some qubes trying to migrate directly from 38 to 40. Then I realized I had to do it 2-step. If that’s what you did, it wasn’t clear from your OP.

Could you elaborate? To be clear, I did not upgrade templates themselves in place, I just switched them.

So if you connect to the qube’s console and login as root and run xterm then the GUI xterm window will open?
Then what happens when you login as user and run xterm? Do you see any errors?

Interesting! If I run it from root session even with “su user”, it works. But if I try to qvm-run as user, it does not (no errors, just silent). On a normal (non-malfucntioning) qube it does.

Did you only try to run xterm using qvm-run?
Can you try to connect to to qube’s console using qvm-console-dispvm, login as user and run xterm there to see if there will be any errors?

Works, no errors.

That’s strange.
Did you start xterm using qvm-run as service or as shell command?

qvm-run --service -- qubename qubes.StartApp+xterm

or

qvm-run qubename xterm

First one gives the same “failed to connect to bus” error.
Second one works.

It should be debian-xterm instead of xterm. Try this:

qvm-run --service -- qubename qubes.StartApp+debian-xterm

And also this:

qvm-run qubename /etc/qubes-rpc/qubes.StartApp debian-xterm

And also open the qube’s console or GUI terminal and try to run this command to see the possible errors:

/etc/qubes-rpc/qubes.StartApp debian-xterm

The templates are fedora. None works except the last one (with xterm, obviously, not debian-xterm). The frist two fail with code 125 and “Failed to connect to bus” error message.

Maybe you have some old files in bind-dirs or something in rc.local?

Nothing in rc.local.

In bind-dirs i have openfortivpn config, cups config and /var/lib/snapd.
Could snap be an issue?

qvm-run --service -- qubename qubes.StartApp+xterm

This does not work, but this does:

qvm-run -u root --service -- qubename qubes.StartApp+xterm

Maybe try to check other dotfiles like ~/.profile or ~/.bashrc.

Nothing suspicious. Also I tried to move entire .local and .config away and it did not help, either.