Building out Documentation on Non-Free code. [Orca Project 1]

User Support
1

Hello everyone! I did not know where to best post this, as it’s an unusual request. I am planning to begin helping to build out further documentation wherever I am qualified to do so (which is fairly small an area right now, but will grow in time). The first thing I’d like to build out, though, based on a discussion on free/libre and non–free software here, is a list of the non-free or proprietary software that Qubes needs to keep up to date on security or to just provide firmware functionality for devices, along with what they do and where they live in the system, and also why they need to be included. It’s not something that will be important to everyone, so please don’t post here about how you aren’t interested in that, if that’s not for you, but some users are interested in the spirit of libre software and open source in the purest sense, and this may be valuable to them. For example, some firmware is loaded into a default debian template automatically that is binary blobs, and this is very useful if you need it for a device (e.g. wifi), but I think it is worth noting that in our documentation so users are aware of that reality and can make a more informed decision about what they want to do, learn more about the state of computing at this time, or even just so they can see that transparency and connect more deeply with the project. I want to be explicit that this is not meant to disrespect the incredible work being done with this project or to somehow suggest it should be a different in some way. Quite the contrary, I think this will add to the value of what is being done by further empowering the humans who choose to be part of this community.

All that aside, since I am a new Linux user, I am looking for some assistance from some of you who are more experienced to give me some direction on how I would determine what software in a template is free OSS or not. I saw that @airelemental had a nice neat display in the forum topic I linked that shows some of this already, so perhaps you may be of some help? I would also like to know in the project in general what closed-source software is needed (like in Xen or dom0) too so I can make note of that and figure out how to best document it. Who might know that information and be able to let me know how I can keep up-to-date on it?

Finally, I am also a new Qubes user and new forum user, and I am going to be building documentation for the first time, so any pointers will be greatly appreciated!! I am reading the documentation about building documentation, but I imagine some of you have done quite a bit of that already and might be able to share some wisdom.

By the way, the [Orca Project 1] is for my own convenience as I plan to do several of these sorts of projects in documentation and I want to keep my brain organized. I can modify that title if necessary of course. Thanks all!

2 Likes
2

I did a quick web search for “debian identify non-free install” and found this unix.stackexchange.com post.

aptitude search '~i ?section(non-free)'

So I ran the above in some of my minimal templates and it appears they don’t contain any non-free packages. So I tried it in the one template I know contains non-free (the one for sys-net) and this was the output:

i firmware-iwlwifi - Binary firmware for Intel Wireless cards

… as expected. So from that follows that:

  1. the debian-minimal templates do not contain any non-free packages by default, although they are setup so a user could simply install them without having to modify /etc/apt/sources.list – that also means that a person who does not which to accidentally install non-free software can simply remove the contrib non-free and be done with it.

  2. the first step of your task is pretty straight forward: take a virgin debian-11 template and run the above command, which will result in you having a complete list of all non-free packages included by default. You could upload that list into a wiki-post and crowd source the “what is this and why is it needed” part.

Let me know if you’re stuck and I’ll do my best to get you unstuck.

1 Like
3

Here is the list for debian-11 as currently provided through Qubes OS:

package description
amd64-microcode Processor microcode firmware for AMD CPUs
atmel-firmware Firmware for Atmel at76c50x wireless networking chips.
firmware-amd-graphics Binary firmware for AMD/ATI graphics chips
firmware-atheros Binary firmware for Qualcomm Atheros wireless cards
firmware-brcm80211 Binary firmware for Broadcom/Cypress 802.11 wireless cards
firmware-intelwimax Binary firmware for Intel WiMAX Connection
firmware-ipw2x00 Binary firmware for Intel Pro Wireless 2100, 2200 and 2915
firmware-iwlwifi Binary firmware for Intel Wireless cards
firmware-linux Binary firmware for various drivers in the Linux kernel (metapackage)
firmware-linux-nonfree Binary firmware for various drivers in the Linux kernel (metapackage)
firmware-misc-nonfree Binary firmware for various drivers in the Linux kernel
firmware-realtek Binary firmware for Realtek wired/wifi/BT adapters
firmware-zd1211 binary firmware for the zd1211rw wireless driver
intel-microcode Processor microcode firmware for Intel CPUs

You can use packages.debian.org to interrogate further.

  • firmware-linux-nonfree is a meta-package pulling in firmware-amd-graphics, firmware-misc-nonfree, amd64-microcode and intel-microcode

  • list of drivers installed through firmware-misc-nonfree

… I think you get the idea.

2 Likes
4

Do you need anything else or are you good to go?

5

Sven, this is perfect, and you are awesome. I greatly appreciate your input because this basically gives me exactly the pointers that I needed to do this. I did not expect such good quality direction, and I want to to thank you for taking the time to look into this!

I am excited to get building the documentation page, as well as continue on to my next one; the delay in my reply has been because I have been busy with my more regular life projects (which will not doubt continue to interrupt me frequently, but briefly) and traveling in between, so I just popped back on the forums yesterday to continue my foray into getting Qubes going as a daily driver.

The only remaining challenge for me now is to take a few hours and figure out how to add to the Qubes documentation (I believe I have found the relevant documentation conveying how to do that), because it is my first time doing “pull” requests (is that the correct term??) in github! So that’s exciting too, but just will take me a day to set aside.

In the meantime, anyone who has thoughts or feelings on what they’d like to see on the documentation, please feel free to post here. My goal is to have this first post attempted by this coming weekend.

6

I do have one question that I think it would be good to have an official answer to, though, as I think deeper into how the page will be built. As always this is an honest question that I want to have included in the documentation (and honestly it might be there already, as I have to look deeper yet), not a question to suggest there is something wrong with the non-free component inclusion in Qubes: How can we trust closed-source binary software/firmware?

I saw that one of the Qubes architects hinted at this in the last post that spurred this one, and he/she said that firmware runs differently and not even on the same processor as everything else, so it cannot interact in an arbitrary way with the rest of the system, and is only loaded as needed by the hardware that calls for it. This makes sense to me, but is there a full official response that should be added to this documentation page? It seems like a good place for it! The motivation for the inclusion of non-free is quite clear already: simply put, we cannot build everything and therefore must trust something to have the functionality we want to have (which includes security fixes, especially in microcode), so that’s not in question. Just the security implications of the trust are what I want to make sure I include on the page.

7

Hint: you might get more answers if you post this on its own topic with that title.

8

I can’t give you official answers, but I can try to clarify the topic for you. First let’s distinguish between three kinds of binaries:

Firmware

This is software provided by an OEM to run specifically on their device / chip and NOT on your main CPU. There are basically three kinds of devices / hardware components:

a) The ones that are purely hardware and do not require firmware (do not have a general purpose processor embedded in them). Beyond very simple components those are rare. Having a processor/firmware allows the OEM to change behavior after the hardware is produced. This can be to fix security issues, general bugs or to customize the component for a specific market.

b) The ones that have a processor and run firmware that is loaded from ROM or Flash chip. In that case your device will work even with an OS that doesn’t provide firmware packages but it also means you won’t receive (security) updates.

c) The ones that have a processor and need firmware but only come with a rudimentary loader in ROM. These devices actually rely on the driver running on your main CPU to provide a binary blob at startup to function at all. I learned recently that e.g. newer Intel audio devices do this. This seems to come into fashion as it allows to reduce the cost of a component by using a small ROM and no (or small) flash memory.

The exception to the “does not run on your main CPU” are of course Intel/AMD microcode updates but everything else above holds true. Intel/AMD CPUs are b)

(Hardware) Drivers

This is also software running on your main CPU. An easy example would be a graphics driver that maps the specific interface of a specific hardware to the general interface expected by your OS and the applications running on your OS.

Applications

This is software running on your main CPU which might use hardware features but isn’t essential for the hardware being able to function or be used.

When talking about firmware your trust choice is whether to use that piece of hardware or not. If you are using it, you are using the non-free firmware that came with it (stored in ROM or flash) or you are not able to use it at all in case of c). Having a conversation about trust is inconsequential in this case. The only difference is whether you are able to receive updates to said firmware from the OEM that made that hardware which you already trust by running it in the first place. This includes your main CPU. There is no free x86 CPU!

With drivers the situation is much better as there are free driver implementations for most hardware and you can make buying decisions to only purchase hardware for which FOSS/libre drivers are available. You (mostly) don’t have that choice with firmware.

Now about “non-free” being enabled by default in the templates shipped by the project: this only means that it is possible to install non-free software by default. One still has to actually install it. The Debian-minimal templates have non-free enabled but do not actually contain any non-free packages. The user has to manually install them.

With the full Debian template that comes with the installer the firmware packages listed previously in this thread are installed by default. All other non-free packages would need to be manually installed by the user. Here it’s important for you to understand what I explained above:

  • Hardware that is present in your system and requires firmware ALREADY runs that firmware whether those non-free packages are installed or not. Installing them simply allows you to receive updates to that firmware or run that hardware at all. There is no trust decision in whether to install those packages or not, but there is the general principle of isolating hardware access in dedicated untrusted qubes (sys-net, sys-usb, sys-audio & sys-gui) and thereby mitigating potential issues with compromised firmware. When it comes to your main CPU and microcode updates you are already running non-free microcode on proprietary x86 hardware. You can refuse to apply the security fixes but how exactly that improves freedom is beyond me. You are just running the vulnerable non-free version that is already on your CPU.

  • If the hardware is not present in your system, the firmware obviously won’t run. Then the argument is whether the packages (that never run) are actually present on your hard disk. I hope you agree with me that’s not an issue at all then.

In summary:

  • installing non-free software is possible by default – this just removes a barrier
  • only non-free firmware packages are installed by default – no non-free drivers or applications!
  • having non-free firmware packages installed simply allows updates/fixes to reach the hardware which is running said firmware anyway
2 Likes
9

Before you spend too much time on this part, it would be a good idea to read through this to check whether your proposed documentation will be appropriate for the official docs as opposed to the community docs:

https://www.qubes-os.org/doc/documentation-style-guide/#core-vs-external-documentation

Another way to put it: Because we have no choice.

1 Like
10

@Sven , I really am gratified that you have offered such a large amount of your time to explain and assist with this, even though it’s not something you personally need/have great interest in. That’s really nice to see in a community and it means I am able to get this done far sooner, and understand far better! I hope I can return the favor someday. Also, your explanation of firmware vs drivers is incredibly clear and helped to clarify relationships in my mind very well. It’s funny: I technically “knew” most of those distinctions before (from the world of Windows administration), but this solidified for me, not only how this all works in Linux, but also the relationship from vendors (like Intel for example) to the OS community. Do you mind if I paraphrase or quote some of what you said for this article/documentation?

@adw , Thank you as well for taking the time to do so much work on documentation in general, as I see you pop up on most communications for this project, and for specifically helping here. I just read that article you linked and I am a little conflicted. I believe that some portions of what I just wrote up should be in core documentation as they are specific to the QubesOS project choices (like where microcode runs in dom0 and enabling the non-free repos in Debain for example, although that last piece might be irrelevant soon based on the Debian community voting to change that handling soon), but then some of what I wrote up is general information about how these things work in the Open source community, a lot of what Sven explained above, and what users can do to use non-free firmware or applications or how they can disable those, and that seems like it should be external documentation to my understanding. I don’t mind at all maintaining the documentation as long as I am comfortable with how to do it, which I will be in time.

Overall, given Sven’s great clarification and everyone’s posts on the original thread that inspired this little project, I really feel like the QubesOS project is making the only reasonable choice it can to accomplish its chief goal of security in the current computing world landscape of proprietary devices (like x86/AMD64, etc) with how it handles these things (and indeed many other GNU/Linux-based distros have come to the same conclusion about how to implement blobs/binaries), but I do want to add to the core documentation and explanation of the QubesOS project-specific choices (what is included and why. there is not much so it isn’t a ton of text) or at least a link to the article explaining how Qubes does set this up and why, and what users can do to assist themselves in making choices that support their personal truths or ideals as much as possible in the QubesOS system.

11

I prefer FOSS whenever I have a real choice. I’m just not absolutist about it in situations where it would harm security or productivity.

Of course not. The whole point of this place is to share.

1 Like
12

Happy to help, and thank you for contributing! :slight_smile:

It sounds like the natural solution might be to separate what you write into two things: a contribution to the core docs and a contribution to the community docs. (I take the latter to include any documentation that lives on this forum).

2 Likes