Yes, it works. It is the Hermes Agent that is doing all the work, and I run it in a standalone VM with full root access, another reason why you want to run it in its own VM.
The inference engine does need to have tool calling enabled, and you need a model that supports tool calling.
The Qwen3.6 models are probably the best models you can use locally for tool calling.
Exciting, thank you!
btw, in my tests, Gemma4 worked best among a number of local models I tested (including Qwen3.6).
Currently, it’s using LM Studio, but I’ll switch it to running
llama.cpp and
Just noticed debian 14 has packaged llama.cpp package[1].
Footnotes:
[1] I Challenge Thee
Yes, it works. It is the Hermes Agent that is doing all the work
I am currently testing Goose Agent[1]. Do you think if Hermes Agent is
better?
I experimented with something like that as well a short while ago. But afer one or two days of updates the nvidia driver segfaulted on start up.
Did you have this problem before?
It might be a good idea to pin it?
I have never used Goose Agent, but Hemes is one of the most popular agent because it works very well.
No, I only install the nvidia cuda driver not the full nvidia driver, and I’ve not had any issues with crashes.
You could pin the driver, or simply not update unless you have a reason to do so.
Tested the guide yesterday. worked fine for two boots.
Started it again today and the driver crashes again.
Didn’t even update. Hope I find time to figure out why.
Edit: Now it works again. Idk. GPU drivers are just broken af.
Do you use the GPU in more than one VM?
In my system using an 4060, the first VM the GPU gets attached to will load the firmware onto the GPU, and the firmware will persist after being detached from the VM. It also persists through reboots, the firmware only resets after the entire system has been full powered down.
If you are using the GPU with multiple VMs, make sure you are using the same driver version in all the VMs, or you might get random crashed depending on what firmware got loaded onto the GPU.
That is maybe related!
Edit: It is not. Every second reboot the GPU driver just crashes X.
Fyi managed to get Hermes Agent (without browser addon) working using a debian 13 minimal template + appvm combo by doing the following:
Can you provide the arguments for llama-server binary ?
The qube doesn’t launch applications after running these commands:
# Download and install the nvidia repository keys
curl -fsSL https://developer.download.nvidia.com/compute/cuda/repos/debian13/x86_64/cuda-keyring_1.1-1_all.deb -o cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt update
# Install the nvidia drivers
sudo apt install nvidia-kernel-dkms cuda-drivers
# Shutdown
sudo reboot
qubes-gui-agent shows Active: failed
UPD: Oh sorry, I forgot to add NVIDIA for this qube in qube manager 
Is it a Discourse bug if we see a different user ID in the quote? It could be a threat (and a privacy issue). cc @moderators
Cheers,
PS : please delete my message & attachements when read. 
I am not totally following this and would like more help understand.
Does this allow GPU passthrough when there is only 1 GPU and then use CPU to display dom0 somehow? I thought this wasn’t possible. Did you find a way around it or am I reading the guide wrong?
Thanks for this guide @anon40243023 .
I’ve began to follow it, but was too hurry, so retried, and still stuck.
VGA and audio pci of the discrete GPU now have pciback, but how to be sure that the GPU has been passed through to ai-ollama VM ?
The only line i get with lspci is 00:03.0 VGA compatible controller : Device 1234:1111 (rev 02), which really doesn’t feel good.
Regards
[EDIT] ok, sorry, it was maybe obvious, but since i just followed the tutorial line after line, I did not noticed I hadn’t attached the VGA & audio to the ai-ollama VM, now it’s clear 
Thanks for the guide - it worked very well 
I used it with a AppVM for paperless and had the issue with the systemd service for ollama (/rw/config/ollama.socket and /rw/config/ollama@.service) that it spawned too many sockets (ss -antup → TIME-WAIT in the ai-ollama VM) and as a result no further communication/socket was possible. The following socat command reuses the connection and worked like expected without further problems:
socat TCP-LISTEN:11434,fork,reuseaddr EXEC:"qrexec-client-vm @default qubes.ConnectTCP+11434"
I guess this can be also implemented as a systemd service.
For those who use paperless with the docker compose setup, you will also need the firewall rule:
sudo nft insert rule ip qubes input iifname "br-*" tcp dport 11434 accept
