Sudden dramatic slowdown, sys-usb refuses to autolaunch at boot, dom0 at ~20-30%

User Support
1

After no particular thing, my x230 is suddenly running like a dog.

most concerning to me is that sys-usb (a disp- version) is now refusing to autostart. I don’t know what the default settings are supposed to be on this qube, but I was surprised to see it is in HVM mode, and even though it has no net-vm, there is a big red warning in settings that says it has direct net access.

I am afraid to force it to start manually.

I think it highly unlikely that a nation-state actor would be interested in me, but the possibility that a hacker might be looking looking for a crypto wallet (but don’t have one…) or something remains. I have no idea how that could happen, though. More likely is that something in this ageing hardware is giving up the ghost, or an update has broken something.

with sys-usb down, i am not sure i can extract a back-up.

other things are:

  • a fedora template (one with extra apps) has been unable to update for days now. Can’t reach a server
  • dom0 is running with 20-30% CPU load right now, startup and qube launch is incredibly slow
  • this qube is using ~50-60% CPU, only running Firefox (although with my usual extensions), I closed an ebay tab and things got better
  • I have updated Tor browser on the template, but a client qube refuses to load the updated version (loads an old version then commences its own update, each instance)

Any ideas or suggestions, or am I indeed looking at getting a new machine?

2

Anything relevant in journalctl in dom0 before the slowdown?

3

Maybe the CPU is being throttled to the lowest frequency? Which can happen if the PSU or (more probably) either side of its connector is broken. I used to diagnose this from the output of xenpm get-cpufreq-para in dom0.

4

If you did any updates recently, there have been a few problems with increased memory requirements for some templates/OS in recent months. It might be useful to verify the memory settings, especially for any HVMs.

I just found one of my sys-net is running at 100% CPU because of low memory…

5

cpu seems fine on your command and ‘cat /proc/cpuinfo | egrep -i mhz’

6

biggest problem is that i can’t understand what is normal or not.

If I go back to the last boot that was ok, I see a bunch of red, including

et': 4141479173, 'use_hotplug': True, 'no_progress': False, 'slow_memset_react': False}} et': 4141479173, 'use_hotplug': True, 'no_progress': False, 'slow_memset_react': False}} et': 4141479173, 'use_hotplug': True, 'no_progress': False, 'slow_memset_react': False}} et': 4018592953, 'use_hotplug': True, 'no_progress': False, 'slow_memset_react': False}}
This seems important?

Also a lot of complainting about

May 09 14:14:23 dom0 (systemd)[4790]: PAM unable to dlopen(/usr/lib64/security/pam_sss.so): /usr/lib64/security/pam_sss.so: cannot op> May 09 14:14:23 dom0 (systemd)[4790]: PAM adding faulty module: /usr/lib64/security/pam_sss.so

Plus some failures in Atomic updates.

For one qube, whonix based, there are yellow lines complaining htat dom '8' still hold more memory than have assigned (numbers of dom change).

but honestly, I have now just been through the journalctl records for boots that were before the problem and after, and my ignorant eyes cannot see anything significantly different. Nothing, for example, that seemed to indicate something suddenly wrong with a CPU or memory.

1 Like
7

I did see a failure to launch all the sys- qubes, but sys-firewall and sys-net clearly got up eventually, but sys-usb didn’t.

8

sys-net and sys-firewall are running, the only HVMs. According to the little window on the qubes system tray icon, their memory use is piddling, like mostly sitting at 0%, occasionally at e.g. 12%.

Is there another way to check this?

9

The 0% must be cpu use, not memory.

The settings for the failing sys-usb will allow :

  • to see how much memory is allocated, on the second tab. You can increase it temporarily, while testing. I think 700Mb is easily enough - but maybe others can advise.
  • to set the ‘debug’ preference for it, which will increase the logging, and should give access to the console when it boots.
  • It is necessary to "Apply’ any changes, then try to start sys-usb again.
10

problem seems to have all gone away after another restart. Flaky.

FWIW, sys-usb and sys-net are sitting at 384 MB and working perfectly now.

1 Like
11

Have to dislike intermittent problems.

Don’t forget that the memory requirement could be significantly larger for startup of the qubes…!