R4.1.1: xentop reporting high mem usage

User Support
1

I noticed that my Qubes 4.1.1 has been running a bit slower than normal, which is odd considering that my machine has 16 GB of RAM installed.

Upon going into Xentools, I noticed that the mem usage was huge. Here are some stats from Xentools:

Mem: 16590712k total, 16454376k used, 136336k free

Dom0: 25.3% mem usage
Sys-Firewall: 22.3% mem usage
Sys-Network: 2.5% mem usage
Sys-Network-dm: 0.9% mem usage
Sys-USB: 1.9% mem usage
Sys-USB-dm: 0.9% mem usage
Sys-whonix: 24.7% mem usage
whonix-ws: 19.4% mem usage

Please note that I don’t have any AppVMs open. The only thing running is xentop in the Dom0 terminal.

Do note that I haven’t modified the templates in any ways, so the amount of RAM each qube is assigned out of the box is the amount they have now.

2

That’s almost a Windows-level of RAM usage.

Do your logs or dmesg say anything?

sudo journalctl
3

Sure thing. I should note that I’ve since shut down my machine, but I’m still seeing VERY high ram usage.

With only dom0, sys-firewall, sys-net, sys-net-dm, sys-usb, sys-usb-dm, and sys-whonix running, I’m using 13.6 GB / 16 GB of ram.

Nothing in journalctl seems to pop at first site. What should I be looking out for, specifically?

4

Errors, or the same log entry several million times. Something that looks like it shouldn’t be there.

Give this a try.

Run the following command in a terminal of each of your open VMs, as well as in a dom0 terminal:

sudo ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%mem | head

That will show you which processes in each VM are the biggest ram hogs.

5

You can change the amount of memory used in the settings of the vm

6

Disregard what I said before. I was mistaken.

My RAM usage in xentop is similar. 99% used, 1% free.

I could be mistaken, but I have a feeling that’s the amount of RAM available to the Xen hypervisor.

7

xentop shows that total memory, used and free, this is shown in the header information. In the detail list view, you can see the percentage of total memory allocated to vm.

Judging from the information provided by the OP, dom0, sys-firewall, sys-whonix, whonix-ws are all using around 4 GB memory each, which is more than needed.

8

Thanks, how much ram should I allocate to dom0, sys-firewall, sys-whonix, sys-usb, sys-usb-dm, sys-net, and sys-net-dm, respectively?

9

Don’t change dom0.

I think usb, firewall, and net needs around 300-500mb, whoinix is going to need more probably around
2 GB.

You can use the command free -h inside the appVMs to see how much memory is getting swapped to disk, if the vm is swapping excessive amounts of memory the memory limit is too low.

10

Thanks, just curious, where did you get these ram figures from?

11

I just remembered people discussing the same subject on the forum.

https://forum.qubes-os.org/t/questions-on-memory-allocations-for-sys-qubes/13045

There are other threads talk about the same issue.

12

Thank you, looks like they had the same problem.

Someone there suggested limiting sys-firewall to 1000 MB. What issues do you think this could cause further down the line?

13

I doubt you are not going to need more than 500 MB, and you can probably get away with using 300 MB, but I wouldn’t use less than 300 MB unless you are in desperate need of the extra free memory.

My sys-firewall uses 270 MB memory, but it also has 350 MB cached data, so the total used memory is around 500 MB.

The sys qubes also have 1 GB of swap space if they run out of physical memory.

14

It seems true that 4.1.1 has far higher memory use than previous
versions, and that configurations that worked in the past are now
broken.

I never presume to speak for the Qubes team.
When I comment in the Forum or in the mailing lists I speak for myself.
15

I can confirm higher mem usage in dom0 but still efficient:

xentop - 14:43:08 Xen 4.14.5
19 domains: 1 running, 18 blocked, 0 paused, 0 crashed, 0 dying, 0 shutdown
Mem: 16635336k total, 10075960k used, 6559376k free

Linux 5.19.9-1.fc32.qubes.x86_64

But, many, many things aren’t looking as expected recently.

For example, at least in my case, xid is changing after each app qube restart, although based on a templates, and that creates huge problems for win qubes using non-dom0 audiovms…

16

If you look at most of these VM’s settings they list a minimum and maximum RAM. This is to allow for memory-ballooning.

Memory-ballooning generally means Xen will assign maximum permitted RAM to all VMs until a VM needs more than Xen has unused (on startup or on demand) and will then take unused RAM inside VMs, zero it, and give it to other VMs.

Brendan

1 Like