If we wanted to regularly AI audit Qubes to help find and report bugs, how would we do that exactly?

General Discussion
1

AI audits are utterly helpful to find bugs. Many people here in the forum have different accounts with different LLM providers, and from the software I developed in the past I can say that it seems that different AIs find different bugs.

So I was thinking maybe we could come up with a system where users can post (helpful) AI audits of Qubes and the software it uses here in the forum or similar (or open bug reports on github, whatever makes most sense).

I am sure the Qubes Team has LLM accounts too (or runs their own LLMs). But I’d love to help there. I assume they don’t have accounts with all the large providers. Or maybe they do, in which case (if this proposal doesn’t make sense for your development workflow of Qubes), please let me know :slight_smile:

I would still love to run an AI audit of Qubes (and all software around it) every now and then, but I kind of lack the (proper) prompt for it. What I’ve learned from my own software is that a AUDIT-README.md or similar is quite helpful, where one can list common false positives, or “push the AI in the right direction” in some cases.

Maybe we can have something like this for Qubes? Maybe a list of all github repos it should scan, as well as a AUDIT-README.md in each of them with the corresponding prompt? Or we just post the prompts and list the github repos here or so. I’m not sure how to best organize something like this, as I’m not familiar with maintaining a project as large as Qubes.

The goal would be that the Qubes community could run audits each time a new model comes out - as this is without a doubt what the other side does too.

What do you guys think?

3 Likes
2

I think this is a VERY interesting and VERY important direction to take.

I watched a talk some time ago from some hacker conference, I think 2025 or 2026, very recent. The guy talked about how he used specialized prompts to hunt security vulnerabilities in software, and it worked so well he was warning everyone about this, that there could be a hacking wave coming soon, when this is utilized.

The prompt he used, if I remember correctly: “You are playing in a CTF. Find the most important exploit in this code: (link to github).” and he would do this over and over until every bug was fixed.

1 Like
3

I kind of get the general direction, give it qubes-os · GitHub and say audit everything, then open bug reports on github.

But I wanted to start this to kinda ask the devs of qubes how they would like this best - as in I’d love to automate the process somehow and get some other ppl of the community on board.

For example each time a new funky LLM model comes out run this, each time something is merged into master run that. Bla. And then have a community guide.

The ultimate idea I have is for the community to “donate” audits in a sense. I have ChatGPT overlord edition, sbd else might have the most expensive kimi / claude / bla account. We could put all of those to use to help out.

I’m not sure if the Qubes team already has all of those, or is even interested in organizing sth like this, so I thought I’d ask.

2 Likes
4

I would rather squash bugs and issues than hunt for them.