It could improve performance. When you run Qubes OS with low memory you might want to control which qube gets how much memory. i.e. sometimes you might run into memory issues when awake your PC from suspend. There are many guides in this community which uses maxmem 0, maxmem=0. You will find many examples of system qubes and disposable qubes which use a disabled memory balancing. It is a commonly used setting when using minimal templates.
My proposed lines of code allows both limiting the max memory or disable the auto-managed memory share. If you see no need for disabling the memory balancer you might want to allow users to set max memory for disposableRAM VMs (which is the default in my proposal).
PS: Next, I will add a vcpus option which makes also sense to me.
We could simple add some date +%s watchdogs in the script but this would only allow us to measure the script execution time for some sections (qube creation / purge process). It will not help us in detecting RAM issues or quantifying RAM performances issues when reaching max RAM / balancing RAM / suspend, awake cases where this could impact performance.
No, I do not have a good and reproducible benchmark for RAM balancing issues. If someone has a good approach to do this please let me know. I also interested to get some reliable numbers. I would like to run some tests on my devices.
Beside the memmax settings discussion. How about some benchmarks on classical disposableVM vs. RAM disposableVM?
Well, yes if you keep all on Qubes defaults (max 4000 MB) you might run into RAM issues (with <= 16 GB) and therefore you might prefer the classical disposableVM. But if you reduce your qubes all to the (minimum) required RAM that is needed you could still benefit from RAM disposable. Consequently, having an option to set a maximum memory makes just sense.
Agree, adding options should not prio #1. Anyways, I do not see a problem - nor a ‘pilling up’ - in adding standard Qubes settings. I am ok to focus first on the creation and removing of disposable RAM VMs afterwards we can follow up the discussion to merge options to the code.
Meta-conversation: the level of collaboration in this topic makes me happy! 👏
Just a quick note from the spectator’s lane to say that I’m genuinely impressed by how this collaboration is going. The ideas collaboration is great, but it’s the quality of the code collaboration that I really wouldn’t have predicted to happen in a forum topic (by that I mean, with no specialized tooling).
Seeing this makes me think: it seems like moving the Community Guides to the forum was a great idea.
Well done everyone, please keep going, I’ll keep cheering
Meta-conversation
Indeed, nevertheless, I am just working on the code and I have to say it is not that straight forward to commonly working on such a code without versioning, diff and merge capabilities. It would be nice to see some very basic versioning and compare capabilities within the thread code viewer.
Or is there a way - an addon - to have a synced code view with a git repository? This would allow us a better collaboration and maintenance of the code itself.
Hi,
I have “cleaned” ALL traces of my old DispVMs (classic and RAM_Disposables) so that everything is clearer for me and I only see the traces RAM_Disposables. So let’s recap to make everything clear
Situation 1:
when I launch a RAM_disposable and close it BEFORE turning off my laptop, I am left with just two files
The file ~/.config/menus/applications-merged/user-qubes-vm-directory-dispXXXX.menu
The file /run/qubes/audio-control.dispXXXX
With the command "qvm-pool", ram_pool_dispXXX is no longer present.
Once the Laptop is turned off and relaunched, the file, /run/qubes/audio-control.dispXXXX is gone. Only the file ~/.config/menus/applications-merged/user-qubes-vm-directory-dispXXXX.menu is still here.
I also tried to do a qvm-kill dispXXXX and the result is identical Apart from the trace of this file, in my case, everything is perfect
Situation 2:
When I launch a RAM_disposable and I DO NOT close it BEFORE turning off my laptop, there are still many traces in the folders:
~/tmp/
~/.local/share/qubes-appmenus/
~/.local/share/desktop-directories/
~/.local/share/applications/
~/.config/menus/applications-merged/
~/.gnome/apps/
/etc/libvirt/libxl/
/var/log/libvirt/libxl/
/var/log/xen/console/
/var/log/qubes/
/var/log/qubes/appvms/
Could this be due to the “#Leave no trace on file system” section not running before Qubes-os is closed? That’s why I tried to close the DispVM with the qvm-kill dispXXX…
After that, is it possible to execute it before?, I’m not good enough to say it
You can tell me "well, close them before turning off your laptop!! grrrr!! "lol. I’m ok, but in the evening I like to listen to music and do a shutdown -h 'time' to turn it off on its own. So what I’m doing right now is to run this command in the DispVM earlier than the one in dom0
I hope this description can help because it’s a very nice script and really useful
It’s a good idea but the topic isn’t specific to your script Everything was said in the thread for the traces in the /var/ folder and subfolders. the traces in the /home/ folder and the ram_pool_dispXXXX come from your script and not directly from a Qubes-os 4.1 bug…
edit: I’ve also noticed that, when I don’t turn off dispXXX and I restart Qubes, it is, of course, still present but it behaves the same way. like an Appvm, as if I had created it with the “qubes tools”. For example, right now, I’m writing to you with the dispXXXX I created yesterday night and I didn’t turn it off before my laptop! lol
Wow, good catch!
I forgot about the “IdleActionSec=10min” that I set in order to try that automatic shutdown thing requested in another thread, so my laptop duly and unexpectedly shut down this morning… leaving two dispXXXX appvm’s which behave as you mentioned.
if you can’t remove ram_pool_dispXXXX with qvm-pool remove ram_pool_dispXXX. Delete your dispXXXX in qubes manager. Then you could remove it with qvm-pool. Then, remove the ~/tmp/dispXXXX. If it doesn’t work, use sudo umount ~/tmp/dispXXXX and you could remove the dispXXX folder
It’s a good idea but the topic isn’t specific to your script
All the traces you observe (except for the pools and ~/tmp/*) are a bug in Qubes OS. A deleted qube must not leave any traces, regardless of how it is created or deleted. The extra mile the script goes in order to clean up is necessary exactly because of this bug. It would be shorter if the bug did not exist. Hence my suggestion that you share the additional findings there.
Unless you don’t observe any traces with conventional DispVMs.
I will still see what can be done in regards to cleanup.
Reworked input parameters. Now you can use all properties as in qvm-create. Remember to edit your old shortcuts (if you have any), because they won’t work after the change.
The cleanup has been moved to a separate function with a trap to hopefully handle potential undesired interrupts. Unfortunately, this doesn’t seem to fix the issue of having qube remnants after a system shutdown while a RAM qube is running. For the moment I have no fix for this. If the OS sends a kill signal to the script, there is no way for the script to complete. The best thing to do is to close the RAM qubes before system reboot/shutdown.
Added a second script. It removes remnants from ANY non-existing qubes (not only RAM based but traditional too). Use with caution!
i @qubist
first, thanks for your work!
I just tried the new 1st script, it works wonderfully, ALL files are well deleted! on the other hand, I can’t change the tempsize with “-s 5G” or with “–tempsize 5G” (maybe I don’t type it correctly?).
In line 43, it’s not “see man qvm-create for complete list…” instead of “see man qvm-prefs for complete list”? (It’s not very important but … )
Otherwise, for the 2nd script, I have a syntax error near the unexpected symbol near " < " .
I think i will the same error at line 75 too
Checked in another terminal while the qube is active:
user@dom0:~ > qvm-pool i ram_pool_rdisp4646
name ram_pool_rdisp4646
dir_path /home/user/tmp/rdisp4646
driver file
ephemeral_volatile False
revisions_to_keep 1
size 5368709120
usage 8192
As you see, the size is 5G.
In line 43, it’s not “see man qvm-create for complete list…” instead of “see man qvm-prefs for complete list”? (It’s not very important but … )
The possible properties are in man qvm-prefs.
Otherwise, for the 2nd script, I have a syntax error near the unexpected symbol near " < " .
I think i will the same error at line 75 too
Both scripts are verified using shellcheck and show no errors whatsoever.
I see no error during execution either.
Please run the problematic script with bash -x <script-name> and paste the error message with a few lines above it. Remember to hide any sensitive info before pasting.
To make sure we are both using the same code, first check:
yes yes, that’s what I typed, it’s just copying where I made a typing error
It doesn’t match what I have, but it’s the same script.
BUT, the problem wasn’t with the scripts but with me!! I was using the “sh” command which worked very well with the other scripts! With the “bash” command, everything is perfect!
Really sorry! it must be tiredness
It’s so cute to see my /var/log/ folder so clean
