Improving Qubes Certification

The Hardware Compatability List was discussed on the vpub yesterday, in regards to it being outdated and how useful(or not) it is to non-users seeking suitable hardware. It was futher said that the real issue is out-of-the-box compatability - i.e. there is a problem until Qubes ‘just works’.

This reminded me of my thoughts on Qubes Certification. I have definitely seen written that the original intention was to expand the Qubes Certifiaction program into ‘tiers’ (‘sometime in the future’).

I feel that the Qubes Certification program could be improved by introducing such tiers - e.g:
Qubes Ready/Qubes Compatible (runs qubes out of the box without issues)
Qubes Essential (works with open source BIOS - i.e.: existing standard)
Qubes Plus (existing standard + improved hardware switch requirements)
Qubes Pro (This would be an improved standard - e.g. stateless - but for now I think effort should be focussed on the above, as the existing standard already only has 3 certified products from 2 vendors who are modifying old hardware).

Why?
Well, first we have to be clear on what the purpose of qubes certification is. My assumption is that it is to benefit the qubes community, but also increase adoption of qubes (as the post which explained the intention to create different levels indicated) - not merely to provide a list of ‘more secure hardware’ for users to purchase.

Assuming the above is correct - the current certification program can be massively improved.
The certification program functions by providing an incentive to manufacturers. Currently, there is little incentive for most manufacturers - as is demonstrated by the current state of the program. I believe that the program is not increasing the userbase as much as it could be - particularly given most new users will not want to buy old hardware at a relatively expensive price.

How can it be improved?
Currently, as discussed yesterday, many existing users have gone through the uncertainty of installing qubes on hardware that may, or may not, work. The HCL is not useless, but has its limitations. My suggestion is to prove a ‘Qubes Compatible’ certification to manufacturers who ensure their hardware configuration works with Qubes out of the box.
This is extremely low effort, and I believe there would be an uptake, as:

  1. Many devices work with qubes out of the box - or with minimal tweaks
  2. Effort for device manufacturers is negligible - we can provide the simple documentation (e.g. xen.efi tweaks, common firmware compatibility issues) - and provide a guide which the manufacturer gives to users (integrate current work on new user tutorial).
  3. There is a clear benefit to manufacturers - we could create a marketing pack demonstrating current uncertainty around compatible hardware and demand for modern compatible hardware; qubes has a growing userbase and increasing traction. The cost for this certification could be relatively minimal.
    (Ideally, we would pilot this with manufacturers whom have an overlapping userbase - i.e. system76, purism, framework, raptorCS, etc - and we could then use this pilot to market to other manufacturers).

This is a positive cycle of growth, the more certified hardware, the more users. The more users, the more certified hardware. For those who would like to see an improvement in the minimum standard (as I think we all would), this cycle of growth would enable us to gradually improve the minimum requirements (e.g. Qubes Compatible would require coreboot/open-BIOS).

Why Qubes Compatible AND Qubes Essential?
Qubes Compatible is necessary, IMO, to increase adoption - raise awareness (and repeat). Qubes Essential - i.e. coreboot/open-BIOS compatible - is IMO essential as many models are now supporting, or looking to, coreboot. System76 for example.

If Open Source BIOS is becoming increasingly prevelant, why the need for ‘Qubes Compatible’?
There are manufacturers whom are not adopting open source BIOS, or are definitely dragging their feet. I believe offering a low-cost option to them, with tangible benefits, would massively benefit the qubes community, by raising awareness and adoption (positive cycle of growth), without harming the integrity/value of the certification program as a whole. IMO ‘Qubes Compatible/Ready’ is clearly distinct from ‘Qubes Certified/essential/pro/plus’. If Qubes can take advantage of this term ‘Qubes Compatible’ it seems only a positive, as people will ask is it ‘Qubes Compatible’, and manufacturers without certification will then have to clarify this e.g. ‘it supports it, but not certified’, which of-course turns most consumers away when they hear such things. (In a way, like thunderbolt certification).

Notes on the current certification process

I am sure this has been raised before - the current process on the website I think is off-putting for manufacturers.
Currently the program, from a manufactuers perspective, doesn’t have much benefit to them.
Seeing that they have to send 2 units which they have to offer in the exact same config for at-least a year is not encouraging.
I understand why this is here, given current qubes team resources - but this is part of the problem I think. Embodied by ‘we will need to charge a consulting fee for more in-depth work.’
The community has a lot of time for Qubes, and I’m certainly willing to contribute to documentation/adapting existing for manufacturers. I think we as a community have the resources to provide enough documentation to manufactuers such that the certification program can certify a ‘model’ of a device and the T&Cs of certification mean the manufacturer can market different configs (i.e. wifi card, screen, BIOS version), as compatible - but they have to ensure they are, and can be penalised if they’re not.
Please see:
No longer listed on the Qubes website's certification page? - Other OSes - Purism community

1 Like

See also: Short list of laptops/desktops that work well with Qubes OS

1 Like

Wait, but this is already exactly what Qubes hardware certification does (and has been doing all along), isn’t it?

I disagree. Currently the only ceritified products are from companies modifying very old hardware, at a price many consider unreasonably expensive (I’m not saying it is, just that is the perception - which is what matters). My choice of the word ‘prove’ is in relation to proving the benefits to the manufacturers - which as per my previous statement, I believe is not being achieved anywhere near as well it could be.

I linked above a very pertinent post. The only manufacturer of modern hardware who has been certified has many issues with the program, (and concerns were also raised by frame.work on their forums) - see: No longer listed on the Qubes website's certification page? - Other OSes - Purism community
Becoming QubesOS certified? - #2 by Kieran_Levin - Discuss - Framework Community

@adw

Unfortunately, they are not certified now (although they were saying they wanted to try again). Concerning the “many issues”, it’s not that serious: Discussion on Purism - #26 by michael

Who is currently responsible for the certification program, is it @michael ?

hi @Quser59. please see the effort fsflover linked to, it is the “Qubes Ready/Compatible” idea you proposed:

as you’ve read from the links that you yourself have linked to, Xen currently can’t run reliably on the latest Intel/AMD processors because they don’t use S3 anymore. hence:

this is not an issue with the certification program but a technical issue, that obviously we (and the whole Xen community) are seeking to address. this is why certification for current System76, Starlabs hardware is not possible. As for Purism, they don’t actually use the latest processors so certification would be technically possible, but our previous experience with them was bad and we won’t partner with them again for hardware certification – you can look at their reddit, user forums, Better Business Bureau, Trustpilot, etc to get an understanding of their priorities and values and how those manifest for their customers. and anyways they already partner with themselves for Qubes certification: hxxps://puri.sm/pages/best-qubes-laptop-is-the-secure-librem-14/

@michael

I have seen this, but this is not what I proposed - is my original post unclear? Unless you are indicating you already have plans to make a ready/compatible certification? But the post states:

community effort independent from and not endorsed by the core team .

Without suspend are there any other power management problems, or is it just s0ix killing suspend support?

Somehow we have gone on a tangent. Yes this is a current issue, but it does not prevent certification fullstop - and I don’t feel like my suggestions have been addressed (are they unclear?).

I was aware of previous issues but wasn’t aware this was the stance, thank you for the clarification.

I think that something is either unclear or missing, which makes it difficult to understand and reply to your proposal. In particular, what actionable steps are you asking the Qubes OS Project to take, and what reasons are there to believe that taking those steps will produce worthwhile results?

@adw Apologies for this, I am working to provide a clear suggestion to you - I will update this post with the relevant information when I have done so.

1 Like

@michael This is quite unfair to Purism. I am not affiliated with Purism in any way and I am not invested in it. I just support and follow their work, because I care about free software, user freedom, privacy and security first and foremost (see my nickname); also a happy user. Same reasons why I support Qubes OS. (I will be happy if you suggest a reasonable alternative, since they are far from perfect. I am not aware of any.)

AFAIK they are the only modern hardware company that promotes FLOSS, security, privacy, and freedom, makes a petition against Intel ME, and many more things. The only modern hardware and software company actively promoting Qubes OS and intentionally designing their hardware to support it. A perfect fit for Qubes OS to collaborate with, one would expect.

What I see instead are self-contradictory claims about previous experience and links to random Internet forums filled with haters (of course, their own forums also have many haters, but quite a few reasonable people, too). I have read most links there and I see many people intentionally or unintentionally misinterpreting what is happening with the company. (Just like many people in the Internet do not understand Qubes OS and think it’s a snake oil until I explain and give some links.)

You contradict yourself, because recently you said on Purism forums:

Are you suggesting that a company intentionally designing hardware for Qubes OS and promoting it is a bad thing?

I hate how two best (in my humble opinion) projects fighting on the side of users are also fighting against each other. This is not normal and harms your users.

well, the intention of my post was to make unambiguous why Librem 13 certification stopped, not to propose re-certification. I considered discussing re-certification with them but then I re-acquainted myself with the current state of the company.

that’s the facts unfortunately.

while we don’t have an explicit “ethical” requirement for official certification, I feel we would do our users a dis-service otherwise. I look forward to us continuing to use Qubes Certification to raise the profile of deserving lesser-known FLOSS-friendly manufacturers.

and as you’re aware, we will have a separate “community-curated recommended devices” list that will include the Librem 14.

we can discuss more in the Purism thread but I have little energy to rehash what is publicly available – at some point someone smart like yourself will read the publicly available information and understand, or not and we move on and put our energy productively elsewhere.

thanks!

Draft proposal is now available at:

@michael
@adw

Thanks, @Quser59. As I mentioned in our private message exchange, I’ve passed this along to the team, but I can’t promise any particular response. Either way, thank you for the recommendations and for caring enough to make them. :slight_smile:

1 Like

yes, thank you for sharing your thoughts in such a well-structured way! it’s clear we have some improvement to make, I will reflect on your suggestions & discuss with the others.

1 Like

Thank you for taking the time to review them @michael .

As mentioned already - Disclaimer:
The creation of the repo is to enable refinement - the current repo is more of a ‘here is my suggestion for what could be done to improve’, instead of ‘do this exactly’.

Is there any feedback of yet @michael ?

The intention of the repo is to leverage community collaboration to reduce the time-burden to the core-team; so any feedback from the team which can lay-down hurdles/requirements I can get to work on refining the repo/offering solutions.

If anybody else wants to contribute: feel free to create an issue or a pull request. I will add the taxonomy ASAP so it is clear how everyone can chip-in.

hi @Quser59, if you were to summarize the issues/problems you see with the current program, what would they be? that would help me understand what you are trying to solve with your proposed draft.

the drafted documents seem to be trying to create “structure” for the program that maybe you see as lacking as someone who is not a device manufacturer? but i’m just guessing since feedback from manufacturers/vendors regarding the certification program are obviously most relevant for us and we haven’t received feedback from them that this was an obstacle for them. (again i shared at the beginning of this thread what is the main obstacle for additional hardware certifications at this time.)

re: better communicating to users what devices Qubes works well with, i don’t know if you saw the latest version of the Community-recommended computers list which we have also now incorporated into the main website.

trying to understand the issues you see with the current process, I re-read your first post. this seems to be your initial assumption:

which is not fully accurate. from the Certified harware page:

The Qubes OS Project aims to partner with a select few computer vendors to ensure that Qubes users have reliable hardware purchasing options. We aim for these vendors to be as diverse as possible in terms of geography, cost, and availability.

but also:

We hope these hardware requirements will encourage the development of more secure and trustworthy devices.

so in addition to trying to provide users with ease-of-use of pre-installed devices, part of the point of certification is to encourage more secure hardware development.

to help users navigate the world of non-pre-installed devices, we now have the Community-recommended computers list.

@michael

  1. Current barrier to certification is too high. Suggestion: revise into tiers, maintaining certification integrity.
  2. Lack of clear terms & manufacturer documentation/guidance is off-putting - given that Qubes certification is currently far-down the list of priorities for manufacturers*.

As I am not privy as to what has been discussed behind closed doors I cannot offer the same insight you have.

However, from the limited public discourse I have seen regarding this matter - from the manufacturers themselves, the apparent sentiment is that the lack of:

  • clear terms/structure
  • documentation
    In combination with ‘consulting fee TBC by qubes team’, is off-putting, given that:
  • no real proven benefit/revenue-add (current case-studies are very expensive legacy-hardware by obscure manufacturers)
  • Qubes is not top-priority

Specifically from Purism, frame.work and system76-(-If I recall correctly).

I am aware of this, however this does not account for the sentiment from manufacturers when this was not an issue - and regardless, this is an entirely separate issue which is in no way inherently tied/limited to qubes certification.

As it happens, I actually wrote a post on github specifically referring this as it encompasses the point most pertinently, please see:

Please see:
qubes-cert-improvements/certifications/why-qubes-ready.md at main · rootnoob/qubes-cert-improvements · GitHub.

thanks for the response, that is helpful.

I haven’t heard/seen such feedback from frame.work or system76. the issue with both is that they run Tiger Lake processor, and framework doesn’t yet have open source BIOS. they don’t list lack of clarity, documentation, costs, in their reasons for why (in the post you linked to). so that leaves Purism, which has been discussed elsewhere.

if you change the definition of certification (like you are proposing) into different categories (which could be confusing to users in diluting the term “certification”), you are still left with the Tiger Lake issues, which can only be dealt with on the Qubes team side. that is, it is not because the manufacturers are not interested (“Qubes is not top-priority”) but because there are technical barriers on the Qubes-side. We have declined offers for certification/testing laptops from some manufacturers because we know they won’t function to user-expected standards yet.

yes but i don’t quite follow your reasoning. we have now done the “hard work” of making this list. no manufacturer is going to do this work for us. if you look at that list, there is little expectation of any of the manufacturers of those computers to want to be certified or care about Qubes – most of those computers are even not sold by them anymore.

anyways i appreciate your efforts. it feels a bit like you have extrapolated from the unique experience with Purism to think that the certification process itself is blocking modern manufacturers from seeking Qubes certification, when i can assure you (and you can see from public threads you link to) that other manufacturers are interested but that technical issues prevent this (for now).

put another way, let’s say we implemented your vision today: would that allow System76, Framework, Star Labs, etc to get “certified”?

the answer is no, because no matter the different levels of “certification” you have proposed, Qubes running on those laptops doesn’t meet basic user-expected functionality right now. System76 laptops can’t get “Qubes x Ready” certification, Framework can’t get “Qubes x Ready” certification, etc. the responsibility to fix that lies with the Qubes/Xen community, not with the manufacturers.

1 Like