Curious about best practices for using qubesOS while also working on some ML projects

User Support
1

My current OS is kind of broken (don’t ask) and I’ve decided to move my PC back to qubes (I already know its mostly compatible as I have done it before)

But I have also gotten into machine learning recently, I was curious if anyone else both uses qubes and does ML, do you set up gpu passthrough, or simply install all the tools directly in dom0?

Personally I’m considering the latter option simply because I do trust the software I will using, and it seems likely easier/more secure then setting up passthrough.

3 Likes
2

After having duckduckgo ml. I wonder which one you mean?

3

I don’t think the first post leaves any ambiguity:

4

Good that you pointed out that I had not read closely enough. My search indicated that ML could also refer to some gaming. I thought OP was referring to two different things. Playing games. and a topic unknown to me, “Machine Learning.”

What i should have said, is what my emotions were telling me about the words “Machine Learning” was the question.

i know I am ignorant, what kind of “Machine Learning” are you referring to? Would I want to do that?

If it is; Taking MIT courses online. I will probably still remain ignorant.

5

I do not really know much about machine learning, but I am using GPU passthrough for hash cracking and it works just fine. The graphic card is automatically attached to the qube when it boots so it is quite transparent. I guess it would be the same experience with machine learning.

You will however need to have a second graphic card (or one integrated within your CPU) to perform GPU passthrough.

6

are there any good links to pass through tutorials/documentation you are aware of? It seems pretty scarce from my looking around a bit

7

Yeah, GPU passthrough guides/posts are spread everywhere :confused:

I mainly used these ones to setup mine:

I took some notes since I prefered patching /usr/share/qubes/templates/libvirt/xen.xml rather than stubdom-linux-rootfs.gz because it seemed simpler.

You can find them below (quickly translated from French to English). I use this technique (and a friend of mine as well) to set up our GPU passthrough and it works fine. Basically you just need to apply a patch to a specific file, set a “qvm-preference” to the qube and then attach the GPU. The rest is to set up Nvidia drivers in the qube, I left it there since you may find it useful.

Do not hesitate if you have problems with the setup.

Infos

Works well but requires a patch in order to use a virtual machine with more than 3GB of RAM.
The Github issue following this problem : https://github.com/QubesOS/qubes-issues/issues/4321

Patch

In order to have GPU passthrough in a virutal machine with more than 3GB of RAM, you have to apply a simple patch in /usr/share/qubes/templates/libvirt/xen.xml (will need to be redone after each update of this file).

In the file /usr/share/qubes/templates/libvirt/xen.xml (remember to backup the file before doing any modification, just in case):

$ diff xen.xml.bck xen.xml
174c174
<                             ,netmask={{ vm.netmask }}"
---
>                             ,netmask={{ vm.netmask }}{% if vm.features.get('gpu-passthrough', False) %} -machine xenfv,max-ram-below-4g=3.5G{% endif %}"
180c180
<                             ,netmask={{ vm.netmask }}"
---
>                             ,netmask={{ vm.netmask }}{% if vm.features.get('gpu-passthrough', False) %} -machine xenfv,max-ram-below-4g=3.5G{% endif %}"

NOTE: the option xenfv is probably not useful.

This patch adds parameters to qemu when the feature (set with qvm-features) gpu-passthrough is set for the qube (it prevents messing with the classical qubes). This is done in the emulator section (that controls qemu)

Then, in dom0, you just need to add the feature to the qube you want to passthrough the GPU:

qvm-features myqube gpu-passthrough 1

Install Nvidia drivers in a standalone qube

Note: it is probably possible to do it in an AppVM by installing the drivers directly in the template and blacklisting nouveau. Blacklist is probably also possible through a kernel cmdline param like modprobe.blacklist=nouveau.

  • Choose a default template
  • Follow the procedure to have the kernel “provided by qube”. Cf. Managing qube kernels | Qubes OS
    TLDR:
    • sudo apt install linux-image-amd64 linux-headers-amd64 grub2 qubes-kernel-vm-support
    • sudo grub-install /dev/xvda
  • Reboot the VM with the kernel “provided by qube” (Settings - Advanced - Kernel: provided by qube)
  • Install Nvidia drivers (from the non-free repositories of debian):
    • Add the non-free repositories if needed in /etc/apt/sources.list: http://deb.debian.org/debian/ bullseye main non-free contrib
    • Install required packages: apt install nvidia-driver nvidia-cuda-toolkit
    • nouveau driver should automatically be hidden by the packages installed in the previous step.
  • Poweroff the VM, and attach the graphic card via PCI passthrough and add it the preference gpu-passthrough like explained in the Patch chapter.
  • Start the VM
  • Enjoy
2 Likes
8

I don’t know much about Xen internally, is this how Qubes actually implements their qvm-features assignments? It would be better if devices assigned to VMs could be checked conditionally to automatically apply patches when a graphics device is assigned.

9

This is definitely a dirty patch to make things work and most likely not the way the problem with >3.5Go RAM qubes with PCI passthrough will be fixed.

It just felt a bit more user friendly to use a custom qube preference to check if the patch is required rather than prefixing the qube’s name with “gpu_” for instance. And it was less time consuming than searching how to do that automatically.

1 Like