I just had a weird incident in which the sum of the RAM in all VMs + dom0 was less than the total system RAM. In other words, some RAM just completely went missing (at least as reported by the Qubes VM Manager). I've never seen this happen before.
Using low-level tools in dom0, I can see that all of my hardware memory modules are still there and recognized correctly by the system at the correct sizes.
This is just an initial report. I'm still investigating...
Hm... I think maybe it's being used as part of the system cache or something. When Qubes first starts up, for a split second, it shows the correct full amount of memory in the Qubes VM manager. I'm just surprised because it's kind of a large amount. If it's percentage based, then I suppose it makes sense.
Check total_memory in 'xl info' output. How much memory is missing? There are
always ~50MB reserved to Xen itself. Also after VM shutdown its memory can be
redistributed after some time (namely: when any other VM change/updates its
memory requirements).
total_memory shows no memory missing. But 10-15% of the total at any given time is still missing (not accounted for) in Qubes VM Manager. Expected behavior?
Does it match free_memory in 'xl info' output?
Pretty much (within a few hundred MB, but I have some VMs running, so values may have fluctuated a little while I was adding).
The Xen FAQ says that free_memory is memory not allocated to any domain (including dom0).[1] So, I take it that this is normal. But why is it desirable to have 10-15% of the total system's memory not be allocated to any domain?
Oh, I think I just answered my own question.
I experimented by opening a large number of VMs. This caused free_memory to drop to almost nothing. So, I think the answer to my question is: "It's not. That's just what happens when you're not using the memory. It's _free_ memory!"