Qubes code audit

Great, it’s a qubes-core-* =D

The biggest file, restore.py is a 2,1k loc

1 Like

:exploding_head: wow, that’s a big file! Seeing how large some of these files can be, I may have to break things up. Where depending on the size of the component we choose a selection of files, audit them, and note them as checked, and repeat the process again with additional files. I want the amount of code to be digestible by participants, so as not to create any burnout and thus disinterest.


Anyone can participate, but we will be laying down some rules of etiquette so as not to clutter up the forum. More info on that will be provided on the outline. We’ll probably setup a Matrix room for basic questions and discussion.

Keep in mind it’s not the place to learn a new language, one should be reading the official documentation for that. We will be providing links in a pinned Resources post where one can read about security for different languages and that would be the best place for you to learn from. Different people with different areas and levels of expertise will have eyes on the code, so don’t feel like you have to know everything. The goal is to put our collective minds together.

2 Likes

I did a quick analysis of loc for the core-* components and main filetypes, which may help breaking things up, matching people to components, choosing the right size to get started with, etc. Because I used file suffixes and the shell code generally doesn’t use them, they’re not included. I used file suffixes to count most languages; shell data comes from: find . -type f -exec sh -c "file {} | grep -q \"shell script\" && echo {}" \; | xargs wc -l

core-admin

  • C:215
  • py:52247
  • Makefile:459
  • xml:323
  • shell:996

core-admin-addon-whonix

  • py:127
  • Makefile:12

core-admin-client

  • py:36511
  • Makefile:260
  • xml:599
  • shell:99

core-admin-linux

  • py:2796
  • Makefile:50
  • shell:992

core-agent-linux

  • C:1710
  • py:2390
  • Makefile:734
  • xml:484
  • shell:5502

core-libvirt

  • Makefile:69
  • shell:60

core-qrexec

  • C:7744
  • py:13866
  • Makefile:324
  • shell:185

core-qubesdb

  • C:4590
  • py:21
  • Makefile:185
  • shell:57

core-vchan-xen

  • C:1706
  • Makefile:39
  • shell:31
1 Like

This sounds like an awesome idea! Although I’m not sure how the less tech-savvy could participate very much.

If you or anyone else can think of how, then I’d be more than happy to contribute.

If nothing else, a good explanation behind each portion of the OS that’s being looked at can lead to a better overall understanding of Qubes. I’ve certainly enjoyed my deep-dives. I know way more about linux now than I did running Ubuntu back in the day.

There’s a lot in the documentation that is either quite vague or outdated. Hopefully this project could also lead to deeper documentation updates.

3 Likes

This is interesting but quite unclear.

The OP talks about “code audit” (assuming all code is available, which is obviously not the case for blobs). Follow-ups talk about “security audit”.

Without a clear goal, it kind of makes no sense.
Even “security” is too broad.

@quantumpacket has been working on clear goals for a security audit.

Though I reckon there could also be value in newcomers just looking over and discussing different parts of the code together without clear goals.

There may be social bonding value in this activity, but “just looking over [some code]” is not of any value.

In my experience, there’s often plenty of value beyond the social, which I can take or leave. However, we’re going with fixed goals atm.

the idea is to audit the code of qubes os to find any vulnerabilities within the qubes operating system. This does not include blobs. This does not include an audit of debian, fedora, whonix etc. Just the qubes OS codebase

1 Like

It’s not just social bonding. If people are looking over the code they will better understand how the system works and be less likely to misuse it. A lot of successful attacks happen simply because humans do the wrong thing even though the system is technically correct.

I agree with the notion that this activity is not an audit, but I frequently run in to people with attitudes that things like social factors are nice but not relevant because only the technical parts are relevant. This is a forum so I don’t have body language, etc to help me interpret your statement. Maybe you’re not dismissive of those things. But I think it’s likely some portion of people who read that comment will interpret it that way and either nudge them towards or reinforce that world-view in their minds, which I would prefer to avoid. :slightly_smiling_face:

the idea is to audit the code of qubes os to find any vulnerabilities within the qubes operating system. This does not include blobs. This does not include an audit of debian, fedora, whonix etc. Just the qubes OS codebase

Vulnerability to what? - Data leaks? Data collision? Anything else? Hardware side-channels? As I said, this needs to be defined. Otherwise it is huge.

Also that code does not exist in vacuum on its own. It depends on other code (including the compiler or interpreter used, down to CPU microcode), i.e. it is vulnerable in a particular context, and if the context is the factor of vulnerability that should be considered too. So, it seems to me quite complex with all the moving targets that make the code functional. I wonder what entity is capable of such thorough analysis.

I think it would be easy to lose some of the benefits by over-defining early on. With a commercial audit, we’d be on the same page - it is huge. With a community like this, keeping options open is also keeping doors open to people wanting to participate, and OP’s definition seems to me like a good balance to start with. If people want to participate with a narrower scope for themselves, I don’t see any reason they can’t. We could try to narrow down the collective scope by polling upfront who’s available with what experience here, or we could get to know that with more confidence over a pilot period, or we could keep it broad which makes it easier if we don’t always have the same people around.

Unlike a commercial audit, I don’t think we should be too worried about providing false assurance that code’s been covered “thoroughly”, or preventing duplicated effort later on. We can set some guard rails (in progress) and go through systematically and track progress, which would all make the results more usable and help future auditing decisions. Getting some actionable findings and fixes, and some more experience with the codebases and working together in the community look like - and this is just my gut feeling after a couple of months on the forum so do speak up if you’ve got a better view - a sensible level of ambition here and worthwhile.

5 Likes

I wonder if there is some kind of code documentation, explaining shortly which code does what.

1 Like

I’m not sure that a clear definition has been given.

The flexible nature of a community project like this is valuable. But structure is also valuable. I want to emphasize again the benefits of creating user profiles that are of interest to the community. This will help people keep realistic threats in mind. And people who are focused on a more narrow scope might notice a problem that doesn’t fit into their scope, but being primed with the range of threats that are of interest would make it more likely for them to notice those problems. And I like to think that most people would responsibly report a problem if they see it even if it doesn’t impact them directly.

I also want to note that I would typically love to participate more in a process like this, but I’m currently working on creating a Guix template for QubesOS while also working on an academic portfolio so that I’ll actually get admitted to school the next time around on top of working a full-time job so I can pay the bills… so I don’t have much spare time at the moment. =(

1 Like

One possibility would be to write a “Introduction to code auditing” which gives full examples of going in and investigating pieces of qubes code, (including things like where to download the qubes source from), and with a little bit of context so they understand what exactly they are auditing. Possibly even write it as a wikiversity book.

This could get more people into the audit group, even if at a basic level. And once they learn more, they might add that to the book.

We may also want to think through how people should report “hey I found this… is it a problem?” to other auditors before flagging it as a actual problem. (I.E. In short, it may be necessary to thin down the false alarms before they get passed to the qubes team)
One option for reporting would be to have people report it to github so that the report can point to the exact line of code in the exact version of the exact package. Another option would be something more customized where people could report what chunks they have audited, and if they found something or not. Then you could compute summary statics like “how many of the files that I said were fine were later discovered by someone else to have contained a problem?”. This would be actual feedback for people who are now learning code auditing.

Just some Ideas

1 Like

I think the forum would be a reasonable place for this kind of discussion to happen. We don’t want to flood the developers with a large number of reports of varying quality. That would end up causing more problems than it solves, because it would take away time that could be spent addressing issues and a dedicated attacker might be able to find valuable reports before the team has a chance to respond to them. I could definitely help with the filtering process and summarizing reports into a concise problem/impact description. This is essentially what I did for most of my ~4 years as a consultant.

I do have a concern about confidentiality. The QubesOS team prefers to keep security issues confidential until they are fixed, but that is in tension with the very idea of having a large community-based audit. I don’t think it would be practical to keep everything confidential and find success in this kind of effort. So… I would feel more comfortable if someone representing the QubesOS team made a statement on that aspect of it before we started.

This sounds like a really valuable tool but also one that would take a lot of work to build. I agree that it would help a lot with coordinating this kind of audit in a decentralized fashion. I also think it would help us find organizational problems (for example, it might turn out that this kind of audit is good at finding bugs within the logic of a single module but consistently misses cross-module bugs) and help newcomers learn from prior work before they get started.

I already asked the team whether they think that this community audit is a good idea, and they said they do. They are, of course, aware that a security vulnerability could be discovered in any kind of code audit. If that were to happen during this community audit, then the auditors should do their best to respect the guidelines for reporting security issues in Qubes OS, but I don’t think anyone expects or wants a community audit to be conducted in secret. I agree that would be both impractical and undesirable.

6 Likes

There has to be some degree of secrecy. What were you expecting to do if you find a security-busting, double-0-day RCE explote? Just post about it on the forum so we can all get our qubes ransacked at once by script kiddies? :grimacing:

Actually, on second thought, yeah let’s do just that. Wild west rule. :horse_racing: :partying_face: If you don’t keep up with community bulletins you get hacked and lose all the loot in your inventory.

My reasoning: anything we find with a community audit is likely to already have been found by their state-funded counterparts (not because we’re bad but because they have money cheatcodes), in which case it is already being employed against the people. I don’t know about you but I’d rather take an L for the team to a script kiddie if that’s what it takes to patch a hole that is otherwise being used by god knows who to spy on Qubes users daily.

Throw Qubes OS into the forge. It can only come out stronger and better.

1 Like

By necessity it wouldn’t be secret - as soon as we include any comms in the audit design then we can’t give them much trust, because we’ve no sensible way of checking community volunteers aren’t state-sponsored or other malicious. I might be wrong, but I think in the Qubes context shouting louder about a vulnerability won’t get it fixed faster, so there’s no reason to shout from the rooftops once the information is technically discoverable. If someone on the audit manages to realize a vulnerability is sensitive before they’ve drawn the group’s attention to it, then it makes sense to report it in the usual way without opening any internal discussion.

What could be bad is the auditors openly discussing whether a vulnerability is serious for weeks without informing the team. If in doubt, better to send a report than not.

3 Likes

Let’s imagine two possible examples:

  1. One of the participants in the community audit discovers a vulnerability, and the other participants don’t know about it yet. In this case, the discoverer can responsibly disclose the vulnerability to the Qubes security team without informing anyone else and without announcing it publicly.

  2. In the course of a public discussion, the participants in the community audit realize that they have discovered a vulnerability. Since their discussion is already public, all they can do is inform the Qubes security team of what they’ve discovered, including the fact that it’s already public.

I took @skyvine to be asking about the first sort of scenario. It doesn’t make sense to ask about the second sort of scenario, because once you’ve said something publicly on this forum, for example, you can’t snatch it back from other people’s inboxes (or from their minds).

By that logic, allowing the code to be open source is also “wild west rule,” because it allows bad actors to discover vulnerabilities more easily. Should we also keep the source code secret for the sake of security? Of course not, because that’s just the old security-through-obscurity fallacy.

There’s currently no such thing as “community bulletins” (unlike security bulletins, which do exist), but let’s imagine for a moment that the community were to start issuing their own community bulletins, perhaps in conjunction with this community audit. Let’s imagine they’re like a community-run version of QSBs. There are several things to keep in mind:

  1. QSBs on their own don’t do anything to your system. It’s the security patches released alongside QSBs that actually patch your system and protect it. The QSB itself is just a text document. It’s purely informational.

  2. Sometimes, QSBs contain instructions for special user actions that are required to address security vulnerabilities. This is relatively rare, but it does happen from time to time. In this case, it’s users actually following the instructions and taking those actions on their own systems that does something.

  3. It’s unlikely that community security bulletins would be accompanied by community security patches. Not only would this require considerable expertise (to actually create working patches), but it would probably be very difficult for most Qubes users to trust the patches. It would probably also be non-trivial for many users to actually install the patches, since the community doesn’t have access to official Qubes signing keys.

  4. This means that the utility of community bulletins would probably be limited to providing instructions for users to follow in order to make modifications to their own systems. This is risky for most users, since following such instructions (e.g., entering commands one doesn’t understand in dom0) could compromise the system.

  5. This also assumes that the there wouldn’t be an official QSB (and possibly accompanying patches) that address the same security vulnerabilities. If there’s both an official QSB and an unofficial community bulletin for the same vulnerability, then there’s no reason to follow the community bulletin. It’s more risk for no added benefit.

2 Likes