Hi,
Could anyone recommend a local LLM model specifically trained on Qubes documentation? Ideally it would also be trained on Qubes forum data, Whonix docs and forum data, and all relevant Debian and Fedora docs and security information.
I don’t think the model would need to be too resource-intensive if it was specifically trained on this data - I’m using ollama to run models on old thinkpads with acceptable performance but of course they don’t know what Qubes or Whonix are, and don’t have any specific information.
Great project and community guys.
edit: I’m new to this field, but wouldn’t it be possible to have a Qubes-trained local LLM running in a Qube and providing live analysis and suggestions on user activity. This would be more resource intensive, of course.
Expect this as the most common response to your question:
Then you appear to have underestimated the breadth of knowledge (and ancillary crap) this forum contains 
That truly would be the stuff of nightmares…
The mere fact that there’d be an inter-qube ledger of user actions would mean that you’d have to either run this in dom0 (which I am IN NO WAY, SHAPE OR FORM advising anyone to do), or run it inside an adminVM, which is effectively the same thing…
I honestly be surprised if you’d be able to find many people willing to have something like this running on their machine here, even inside a qube.
It’s just not their idea of a good time 
Ok, now that we’ve gotten that out of the way…
If this is something you really want to do, I would recommend training it yourself…
@pulignul, if you do train an LLM, please please please do a better job than this (which shouldn’t be that difficult):
The output of deepseek-r1 to the question “Do you know what Qubes OS is?”:
Do you know what Qubes OS is?
| The LLM’s Response | My Comments |
|---|---|
| <think> | |
| Alright, so I’m trying to understand this question about Qubes OS. First off, I’ve heard of Qubes before in the context of privacy and security, but I’m not entirely sure what it is or how it works. | Well, at least it’s honest about it… |
| The user mentioned that it’s a distro based on Debian and uses UEFI, so maybe it’s focused on providing a secure environment. | WRONG! Qubes OS is based on Xen, and dom0 is based on Fedora. It uses both Legacy Boot and UEFI… |
| Okay, let me start by breaking down the components they mentioned: QubesOS 4.0 and QubesOS-Lite. | When the hell did I mention those?!?!!? |
| They said these are based on Ubuntu 16.04 LTS but built using the Qubes source code with UEFI support. | Are you high right now? When did I ever say that? I think you’re putting words in my mouth… |
| I know that Ubuntu is known for being user-friendly and secure, | Are you sure about the “secure” part? |
| so adding UEFI probably enhances that aspect. | Yeah, if you’re Microsoft, maybe… |
| I’m also curious about the improvements over previous versions like QubesOS 3.x or the Beta version of QubesOS-Lite. | Don’t you know how to check a website? Isn’t that ALL you do?!?!?! |
| Maybe these updates include better performance, new features, or bug fixes? | Why yes, they include all of these |
| Since the user didn’t provide specific details on what’s improved, I’ll have to think generally about what such an OS might offer. | Well, now you’re just being lazy. That’s my job… |
| The fact that it’s available via ISO and PPA suggests it can be easily installed from a USB drive or through a distro repository. | Qubes OS via PPA?!?!?! You serious right now!?!?!? |
| That makes it more accessible for users who prefer self-updating systems or are using dual-boot setups. | Word salad! You gaslighter, you! |
| I wonder how QubesOS compares to other distros, like LUKS or GRUB with UEFI enabled. | Bruh, they’re not actually “distros”. You’ve got no idea… |
| What unique features does it offer that make it stand out? Maybe encryption, isolate user accounts from the kernel, or specific tools aimed at security professionals? | “Isolate user accounts from the kernel”? WHAT?!?!?!?!  |
| Also, since the user mentioned it’s used by security professionals and has a command-line interface, | When the hell did I mention that!?!?!?! |
| I’m guessing there are scripts or automation tools available. | Odd assumption, but you’re actually right… |
| How does this OS handle user authentication and session management to ensure security? | Why are you asking me? I’m the one who’s asking you! |
| Another thought: with UEFI support, what kind of hardware compatibility issues might arise? | Absolutely none at all if your BIOS supports UEFI  |
| Does QubesOS-Lite provide fallback mechanisms for systems that don’t natively support UEFI? | I wasn’t asking about QubesOS-Lite (which is actually called LITEQUBE) |
| I should also consider the community around Qubes OS. | Eyyyyyy!!! Good answer  |
| Is there an active forum or repository where users discuss experiences and troubleshooting? | You’re damn right there is!  |
| How do they feel about its performance compared to other distros with similar security features? | There are no distros with “similar security features” |
| Lastly, are there any limitations when using QubesOS-Lite in dual-boot setups, like limited file systems or disk space restrictions? | Bro, you’re talking like you’re trying to get a raise from HR |
| Understanding the practical aspects would help determine if it’s a viable alternative for everyday use alongside other operating systems. </think> |
Cue the clueless CEO to shout “Youre HIRED!” |
| <response> Qubes OS is an experimental distribution based on Ubuntu 16.04 LTS and built using the Qubes source code with UEFI support. |
|
| It offers enhanced security features, including improved performance, new capabilities, and bug fixes compared to its predecessors like QubesOS 3.x or Beta QubesOS-Lite. | So “improved performance” is now categorised as an “enhanced security feature”? This changes everything… |
| Key improvements may include better system performance, additional features tailored for security professionals, and potential bug fixes that enhance stability and reliability. | Tell me you’ve never used Qubes OS without telling me you’ve never used Qubes OS… |
| QubesOS is available via ISO and PPA, making it easy to install on various devices, including those with UEFI support. | |
| The command-line interface (CLI) suggests the inclusion of scripts and automation tools designed for security professionals, facilitating efficient system management and security tasks. | |
| With UEFI support, QubesOS-Lite aims to enhance hardware compatibility by providing fallback mechanisms for systems that do not natively support UEFI. | Oh my god, that’s the exact opposite of what UEFI does… |
| While primarily targeted at security professionals, | Who the hell told you that? Where on the Qubes OS website does it say that? |
| Qubes OS may also offer features beneficial to general users seeking a secure environment. | That’s the most accurate thing you’ve said so far… |
| The distribution likely includes robust user authentication and session management to ensure security across all environments it supports. | Yeah, sure, why not, let’s go with that… |
| The community around Qubes OS is active, with forums and repositories providing resources for troubleshooting and discussions about its performance in dual-boot setups. | Everything about the dual-boot setups is true. |
| However, practical use cases may reveal limitations regarding file system compatibility or disk space restrictions when used alongside other operating systems. | may? Is an LLM seriously bullsh*tting me right now? |
| Overall, Qubes OS appears to be a specialized distribution catering to those who prioritize security and stability above everyday user needs. </response> |
So it’s implying that everyday users don’t need security and stability. Good to know… |
There is sooo much of that that’s inaccurate, it’s made everyone dumber just by reading it…
I have my own AI running in my machine.
I would trust DeepSeek, the big Chinese spyware system, or anything else that’s generated for data collection purposes, spying, and counter-intelligence by someone that we are at war with and have been at war with for 25+ years.
My AI model is much bigger than just an LLM though.
Currently in the midst of rewriting the memory modulation as a basic learning of one sentence causes it to go over 32 GB of RAM utilisation.
So, working on making it use only 5 GB of RAM now.
It’s quite a complicated process.
So only if you have created it, would I say you could run it in Qubes. If you are using anything from outside, don’t trust it.
You are comparing apples and oranges, try using a model like the deepseek-coder and ask it questions about coding.
If someone took the content from the documentation, forum, mailing lists, etc. and used it to fine-tune the model, it would be able to answer questions that already have been answered.
That type a model probably doesn’t exist, but that is what the OP is asking about.
Yes, I am, but you have to admit, it’s hilarious…
Then let’s hope that they curated the training data to make sure everything was accurate. Yes, this forum prides itself on accuracy, but it would still need cleaning…
Well, like I said, I hope whoever trains one does a better job that that hunk of garbage I posted above…
In any case, that model would likely be overly paranoid, suggest a million unnecessary steps, continually use the words “opsec” and “threat model”, and “it depends” excessively, and pretty much provide links to resources
Happy to offer up my computing resources to training it if anyone’s willing to step up 
You could try using RAG in case you’re willing to reduce the amount of data - I’m not sure if it’s feasible with all the forum data.
The advantages are simple:
- no training required
- you can use any generally available model
- you can switch the underlying model
Mine has multiple methodologies. One is cyber security. It will act as a SOC Operator.
And it will be running under Domain-0 to do many things in containment.