is it possible that the root cause is related to VPN kill switch feature ?
sometimes i have similar issue, sometimes can / cannot connect,
because i forget if the VPN kill switch feature is on,
so i don’t have connection while VPN sometimes cannot connect.
@newbie I don’t use VPN (anymore).
Things I have tried:
Results:
Next steps:
I am a bit rattled at how quickly my beloved stable setup went to utter useless in a few days. Need to chew on that. Happy side note is that if you have your setup well documented (like I do) switching from debian to fedora or to another version is trivial and not nearly as involved as you’d think.
Have these words stuck right above your display.
Then you’d know if it’s about the hardware if outcomes would differ?
not sure whether a good alternative solution,
but maybe you want to try,
try to create 3 sys-net & 3 sys-firewall,
Short update: I’ve downgraded to R4.0.4 to make sure it’s not hardware related and that I haven’t just imagined “everything” being so much smoother and reliable in the past. After a month of daily usage I can report…
| Qubes OS | R4.0.4 |
| Xen | 4.8.5-42.fc25 |
| Kernel | 5.4.190-1 |
| Templates | all debian-minimal |
| Filesystem | BTRFS |
| qube type | memory |
|---|---|
| sys-* | 250 MB |
| web app | 400 MB |
| 500 MB | |
| web | 1000 MB |
| windows | 4000 MB |
startup time of web qube (buster): 5.05 - 5.25
startup time of web qube (bullseye): 5.36 - 5.59
I am sure now that I didn’t just imagine this T430 being the most stable, fluid and secure system I’ve ever had. What has happened (subjective and specific to my behavior, hardware and environment) is a steady degradation of the experience. It wasn’t that “quickly” as was my initial impression. First things needed more memory and CPU (R4.1), then the machine started freezing during updates and finally I landed in the hell described in this thread. After working from home since January 2020 this last issue started happening on the first day of my first business trip in 3 years. It was horrible timing.
Anyway. I am still not able to put my finger on what’s causing all of this. I have restored a sane environment for me to work, but it’s EOL. Luckily I have two identical machines and after making sure they are both working the same, I will then advance on one of them back to R4.1 and report back here what I am seeing.
If the problems don’t happen anymore we can close the case as “Sven doing something and a reinstall fixed it”. If that’s not the case, I’ll start creating qubes-issues and provide debug information as request. Finally I (we) must face the possibility that these old machines are just no longer supported well (by Xen and maybe by extension Qubes OS). If that’s the case and other T430 users see the same thing it might be time to remove them from the recommended list.
Thanks for an interesting summary @Sven. I look forward to seeing the
results of your test.
I share your feeling that 4.1 is not a good fit with the older machines:
this is disastrous given that they remain the only certified hardware
that Qubes has.
The only way that I have been able to get anything like decent
performance on 4.1 on a range of x220 and x230 is by allocating more
memory to each qube, scaling back on the number of concurrent qube
sessions, and reaping qubes almost as soon as I am finished with them.
On 4.0.4 I was able to keep unused qubes hanging around in case they
were needed, which made for a much better experience.
I notice significant performance hits when new qubes are started and
when qubes are shut down. My feeling is that memory management is less
performant on 4.1, and this is particularly noticeable at these
transitions. I think this accounts for the major risk of a system
crash during updates when a large number of qubes are cycling state in
succession. I have been able to reduce the risk of this by extending the
update process with pauses between each cycle.
There are a few main differences between R4.0.4 and R4.1.
Any of them might result in the instability. Some of these are more or less easier to bisect than others. For example, it’s easier to test different dom0 kernels than Xen versions. If unfortunately, R4.1 were confirmed to be unstable on X230 with debian-minimal templates, then I would suggest start building things and installing them to test. Maybe the workflow of openQA can be used for reference.
This is well known.
It has been reported (and this was my experience) that a clean install
of 4.1 did not evidence these issues, and that they developed after a
series of updates.
I’m still rather suspicious of the changes in/to the newer xen memory grants infrastructure leading to much instability…perhaps through memory fragmentation…or something else.
B
I’m back on R4.1 for a bit over a week and things are stable.
I can’t observe the original issue of this thread nor any crash/freeze. My conclusion is that in my specific case the root cause more likely then not was in my version of coreboot/heads I build myself (v0.2.0-1150) from the osresearch/heads repository. Once I switched to v1.4 of the Nitrokey/heads repository no more issues can be observed:
| Qubes OS | R4.1.2 |
| Xen | 4.14.5 |
| Kernel | 5.15.103-1 |
| Templates | all debian-minimal |
| Filesystem | BTRFS |
| qube type | memory |
|---|---|
| sys-* | 250 MB |
| web app | 400 MB |
| 500 MB | |
| web | 1000 MB |
| windows | 2048 MB |
startup time of web qube (bullseye): 6.70 - 7.09
Is that 6 MINUTES? Just to start a single qube?
It’s seconds. 
If it were minutes I wouldn’t be so casual about it. To compare these are the times I see on R4.0 using the exact same hardware:
startup time of web qube (buster): 5.05 - 5.25
startup time of web qube (bullseye): 5.36 - 5.59
Phew
hello sven
Famous last words.
Freeze happened twice this week. Journal says the last thing that happened before the last freeze is systemd-tmpfiles-clean.