I am in the process of migrating my sys-qubes (-net, -firewall, and -usb) into debian-11-minimal templates. Thus, I have been examining the existing templates for these qubes. On this topic, I have a few questions:
Why are the default startup memory values set to 400 MB and 300 MB for sys-net and sys-usb, respectively? Are these values an educated guess? How are these values estimated, that is, how can one decide that 300 MB startup memory (with no memory balancing, so that value is the ceiling value for the sys-usb qube) is enough for a USB hub (that is, sys-usb)?
I noticed that by default the sys-net is exempt from memory balancing, while sys-firewall makes use of it (up to 4000 MB RAM). Why is this difference between those qubes? Afaik, sys-firewall is connected to sys-net qube like two pipes being concatenated one after the other. So, if sys-net is limited to 400 MB RAM while sys-firewall can memory balance up to 4000 MB RAM, isn’t the whole chain of these two effectively limited to 400 MB RAM?
If you are trying to save memory, fedora minimal may be a better option, in my limited experience. I originally built sys-qubes with debian minimal (debian is my preference) but when I built equivalent qubes using fedora minimal, these used slightly less memory. The starting values are educated guesses but you need to consider the fact that reducing the startup memory may lead to paging/swapping and that may not be a good thing for your SSD in the long term. In order to see what works for you, I would start with some value, run some load and see the memory footprint, both in memory and in swap. You can also reduce swappiness to 1 (or even 0, if you don’t mind the occasional OOM killer triggering at some point) to reduce the wear in your SSD (assuming you use an SSD).
Sys-net needs to have access to hardware, so it runs HVM, which does not support ballooning/balancing. Same applies to sys-usb. Sys-firewall, on the other hand, is a standard pvh, so balancing works well with it.
No, your analysis about the effect of the memory limits in those two vm’s when they are connected in series is not correct. The use of memory is very different. Most of the activity that sys-net does happens in kernel space and usually memory utilization doesn’t increase much over heavy traffic (exception being when you use sys-net for Dom0 updates too). Sys-firewall, in the other hand, could be running proxies or other user space services that take additional memory, but 4,000MB is still a very generous amount and that memory is usually never used and reclaimed by Qubes if there is memory pressure somewhere else.
I don’t want to dabble with swappiness settings. I would prefer least amount of changes enacted on the QubesOS by myself, as I am fallible and my information is limited on Linux OS stuff.
So, what would be the “appropriate” value for the max memory limit for the sys-firewall (considering the fact that it is based on debian-11-minimal). Would you say 1,000 MB is fine?
Edit: I am aware of the Mirage firewall, which sounds fantastic (I remember we have chatted on that before, too). However, currently I am taking things slow and deciding against moving MirageOS’s ISO to dom0 and installing it from there. I would much rather wait for the official support for its installation by the QubesOS team.
Here, QubesOS is doing a full system update. And I am seeing that sys-firewall is using ~1500 MB of RAM, and sys-whonix is also using the same amount of RAM.
So, what would happen if I limit sys-firewall to 500 MB as you guys did? And if I disable memory balancing for it, is sys-firewall going to use swap space? Or is it going to crash due to out-of-memory situtaions?