Plans to Add Search/Filter Functionality to the HCL?

Are there any plans on adding search/filter functionality to the web HCL list? I feel like it is getting quite long and complex.

@adw / @sven this is probably for you.

Search: just press Ctrl+F and start searching (this is a feature of your
browser).

@adw recently implemented sorting, so you could sort by e.g. the release
and narrow down that way.

An actual search/filter functionality on the website would of course be
desirable. I am not a web developer, so waiting for me to do something
about it is not a good strategy. Someone else from the community could
step up an build it and it would certainly be very welcome.

In the meantime, you can access all the actual YML
files
on github and do with them
as you please. I’ve never done this, but in theory one could concatenate
all of them together and then run through a YML->CSV converter and get a
spreadsheet representation of the list.

I might try the above and if it works well in an automated fashion we
might make the CSV available on the website.

2 Likes

A js-based search could be useful. Something like

https://meta.discourse.org/docs?search=search

1 Like

We’re always open to good PRs that enhance HCL page functionality.

As @Sven mentioned, the columns are already sortable. Simply click on the headings. Also open to PRs that make this more discoverable, if that was the problem.

2 Likes

Do people really want search, or filter?
What would be a concrete useful example?

When I was trying to decide what hardware to buy for a new company (~1 month ago), I found the HCL unnecessary difficult to handle (Of course there is Ctrl+F and github and grep and whatnot… but that is clearly not the point here). Since I am an IT-Sec professional (motivated) and an engineer (somewhat knowledgeable) I was not discouraged by these and other problems. (Other people might be.)

When buying hardware which is not on the HCL, one automatically becomes a tester, whether they like it or not. I did not want to be a tester (not saying that I don’t happily give feedback and information where needed); I want to be a power user (as I want to spend my time helping people to improve their security, not tinkering with different ISOs and kernel parameters in hopes not to have to return the hardware or to install a different OS). Presenting the HCL information in a clear and reassuring manner will go a long way in giving potential users confidence in that their setup will working (and that they are not wasting their time and money).

To give useful examples I would propose the following changes (to be discussed):

  • Give a visual cue that the columns are sortable. (I did not realize they were until @adw mentioned it.)
  • Make the columns filterable. (People will have multiple criteria when looking for information. I know I did.)
  • Split the model column into a “manufacturer” and a “model” column (Because filters.)
  • Some “smarter” way to visualize duplicates with different configurations (e.g. features not working in 4.0 but working in 4.1)
  • Add some kind of “age” column. (I have not memorized lenovo’s hardware portfolio, I just want to find reasonable new hardware to buy.)
  • …?

I think there will not be a PR until we agree on what changes we want.

PS: I think we should also look at ways to fully automate the submission and digestion of HCL reports, but that is a different cup of tea. (But these improvements could help to enable that.)

I know a thing or two about Gitlab’s CI/CD funtionality. If Github has something similar I would love to give it a try :slight_smile:

Thanks Tie-fighter, that really helpful to me.

It look as if you want a filter, not a search.

I don’t know what best practice suggests for showing that columns are
sortable - randomly clicking seems to work for me. Paging @ninavizz

Is the “age”, the age of report or the age of hardware? How would the
latter be determined. Is it relevant? Why?
What about other factors that are relevant in Qubes - e.g. maximum
installable RAM? What about cost? That’s obviously relevant to some
users.
Which of the many possible filters should be supported? If you
filtered by (reported) manufacturer, what about cases such as
Clevo/System76/Vant/all the rest , or rebadged Lenovos or Dells.

I’m not clear that there will be features working in 4.0 that wont work
in 4.1. There were for the 3.2/4.0 transition but the 3.2 reports are
still useful.

This is a really difficult topic to approach, with many questions to
resolve, but I don’t think we should put off PR.
Making it clear that columns are sortable would be an advance, which
could be done immediately, while the other questions are batted about.

1 Like

Sortable: Something similar to e.g. wikipedia (Comparison of Linux distributions - Wikipedia) could work well. But this can hardly be a new problem; I am sure there is some consensus among UI designers how to handle this.

Age: I think both are relevant. A report should definitely have a visible timestamp. Maybe filtering by Qubes release would also do the trick. Because how many people are going to install EOL/unsupported versions? Some way to retire/mark “old” HCLs should exist.

And you are right, determining the hardware age is somewhat difficult. There are e.g. still Sandy Bridge (Launched 2011, discontinued 2013) setups in the list. Per se not a problem, but I am not sure those are even available commercially anymore. Having those next to current hardware without any visual cue feels kinda weird.
Maybe making the processor a separate (filterable :wink: )column would satisfy the “hardware age” case.

Manufacturer: The HCL reports already have a “brand” field. How about just using that?

Duplicates: Consider this for example:


Or this:

I think some people might find this confusing.

The problem is easy enough to describe:

A user wants to find a computer they can purchase that will work well
with the current Qubes OS revision.

The solution is much more difficult and includes availability, budget
and whether used machines are an option.

I am all for making the HCL list better, but think that what we need in
addition is a short, frequently updated list of currently available
off-the-shelf notebooks and desktops that run the currently released
Qubes OS well. …but how to get there? Maybe a forum survey every 6-12
months?

One problem is that when hardware starts to be well supported it often
is no longer commercially available. When I got my ThinkPad P51 it
required several workarounds to get R4.0 running (UEFI, WiFi). R4.0.3
and later installed without the need for any workaround or tinkering but
at that point one couldn’t actually buy a new one.

Now I am using an upgraded, maxed out ThinkPad T430 … by far the very
best Qubes OS machine I’ve ever seen or heard about. Everything works
great, the performance is awesome and it uses Coreboot/Heads. But, what
use is it to list such a setup? It requires you to hunt down all the
parts and build it yourself or shell out big Euros for a NitroPad (which
is already listed as a certified option on the Qubes OS website).

2 Likes

Yes to everyone!

I think there are three very astute and opportune solutions here, that could help everyone. I also observed while editing the docs recently, that the HCL page was either somehow buried or not named as intuitively as possible. Which I don’t say to “ding” anyone or anything, it just came to me as a “huh, this could be done better!” while doing other tasks in a similar context. Pretty sure that either Andrew or myself resolved that.

  1. Yes, updating the table with sortable headers is a great idea. Generally, traditional tables are not my own preferred way to enable sorting—but my preferred methods I feel are more complex, than the value this feature would offer the page, given user needs.

  2. It strikes me (based on what I casually observe in the Forums) that the primary need Sven hit on the nose, is a list of off-the-shelf options folks can just go buy. While there is a strong DIY ethic in the Qubes community, that still seems to be a common need among those not yet sure they’re ready to make that big time investment in building out a dream-machine.

  3. Sven additionally adding the “Content Lifecycle Management” angle to such a list, I also +1 with a chefs-kiss. Yes, a Forum survey every 6-12 months, seems like a fantastic idea to maintain the relevance of such a list—and, it nicely removes burdens from the core team, from having to maintain “yet another thing” that could bite the project in the arse the second it becomes out of date for a vocally frustrated prospective user.

2 Likes

Hi @Nina, the sorting was recently implemented by @adw, but it’s not
discoverable. One has to know to click the column header. Maybe you have
an idea how to surface this better?

I also strongly agree that the other shorter list needs to be community
driven. Not only for the effort but also as you point out to not be
“bitten in the arse”. Even thought the certified hardware page clearly
states “We take no responsibility for any vendor’s manufacturing,
shipping, payment, or other practices;” … folks just make an
association and come to the forum to complain when their (sometimes
unreasonable) demands are not met.

Being born and raised in Germany I crack up every time someone comes to
the forum complaining that they sent an email on Friday to a German
company and haven’t gotten any response 48 hours later!!! lol …
different cultures :wink:

2 Likes

I am wondering: How much is compatible hardware a Linux problem and not a qubes problem?
e.g. I have some sound issues that should probably be fixed upstream.
How many are really qubes related problems?

How do you like the idea of marking or moving old (< R4.0) HCL reports? Maybe sort them into directories (according to qubes version) on Github?

PS: Also can somebody please explain to me, why there are .gitlab-ci.yml files on Github? I would like to contribute, but need some orientation.

PPS: I ran the numbers, ~61% or HCL reports are for old/unsupported versions:

$ grep -h -zoP "qubes:.*\n\K.*\n" *.yml | sort -n | uniq -c | sort -nr
    270       R4.0
    189       R3.2
     65       R3.1
     41       R3.0
     37       R2
     30       R2B2
     28       R2B3
     21       R1
     17       R4.1
     17       R2rc1
     14       R2B1
      6       R2rc2
      3       R3rc1
      2       R4.0.1-rc2
      2       R4.0.1
      2
      1       R3.1
      1       R4.0.4
      1       R4.0.2rc1
1 Like

I don’t like it.

It’s a complete mistake to assume that the reports only apply to that
release.
I doubt that people updated a report when they moved from 3.2 to
4.0, but there must be many such users.
The older reports contain useful information which will inform a buying
decision. (Certainly for 3.2 reports, and negatively for earlier
release reports.)

No doubt, the HCL could do with improvement - I suggested some areas
where it could be usefully extended, but they would need much more
research and work.

I don’t understand this idea of “commercially available”, or the desire
for shiny new things. I’m trying to be polite, but your privilege is
showing.
Not every user is in a position to spend large amounts of money on new
hardware, even if it were available. We shouldn’t make those people feel
like second class citizens. They may need Qubes more than you.

@unman:
I am aware that older reports might give insight into what might work (just like similar configurations/models). That is nice to have but hardly the main point of the HCL, is it? From my understanding (and expectation) it is about reporting what hardware has been proven to work. Users should be encouraged to upgrade to newer releases and to create a new report after successfully upgrading (and the effort to submit a report should be as low as possible while maintaining opt-in and consent).

Also I never suggested deleting old reports, I merely suggested marking them or separating them from the current ones. I agreed that my initial idea of hardware age was not worthwhile. That is why it might be better to talk about release version to give the HCL a “sense of recency”. Surely you agree that there is no point on installing EOL versions?

I am happy to answer any questions if there is any misunderstanding. You know nothing about me, so please don’t go ad hominem; this gets us nowhere.
What I am trying to offer is a less niche/hobbyist/engineer and more professional/day-job/user/company oriented view. Surely these should not be treated as second-class citizens either.

I think qubes is really great and would solve a lot of problems and needs of a lot of people. That is where I would like this to go.

Looking forward to hearing more opinions.

I’m in danger of being reprimanded again, but you seem to have
deliberately misread or misinterpreted what I said.

The HCL is not about reporting what hardware “has been proven to
work” - it’s a compilation of reports about hardware compatibility with
Qubes, good and bad.
Reports from even the oldest releases provide information about some of
the security features of that hardware. It has nothing to do with
installing those versions.

My criticism was not ad hominem, nor was it aimed at you personally.
It was a reminder that there are users for whom those older cheaper
machines aren’t a choice. Talking about “off the shelf” and
“commercially available” options is a distraction. (If it needs
to be said, this has nothing to do with DIY hobbyists.)

The HCL is a tool - it can be improved, and I’ve pointed out some areas.

I’d be strongly opposed to a list with “off-the-shelf” options, if that
means anything other than “can be bought”. The list should contain
older and cheaper hardware: who could object to that?

Agreed. Going forward let’s talk about “can be bought”, which is
precisely what I meant when I wrote “off-the-shelf”.

I have pointed out several times on the forum that users will be far
better of choosing hardware that is at least 2 years old.

We need a list of laptops and desktops that:

  1. can be bought used or new
  2. allow Qubes OS to be installed without workarounds
  3. provide a usable level of performance for every day tasks

@unman would you agree with these 3 requirements?

I thought about going through the HCL list picking machines that run
R3.2 or later and have no comments – but that doesn’t really guarantee
that there are no workarounds needed. Rather I think we should have a
thread to collect candidates and then drill down a bit on a few of them
that can be backed up by more than one person. Makes sense?

@deeplow I’d like to start a thread asking for candidates that fit these
requirements and then refine it from there. Which category do you think
is the right one (I wouldn’t want to use “HCL reports” for that please)?

Maybe “General”?

1 Like

Yes, I guess General Discussion with a short and appealing title will do the job.

“Maybe you have an idea” can be a dangerous thing to say to me. :wink:

Issue created here!

Also, FWIW: My email/contact info are on the Qubes website, and I get a non-trivial volume of emails from folks asking “What can I just go to a store and buy that will be compatible with Qubes.” Privilege is certainly a factor, but America is also a consumer-behavior driven society. So, much as I also like to throw down with privelege-checks, I’ve found that in my work with both SecureDrop and Qubes, a bigger problem (in a way) is that much of society has been trained to be spoon-fed everything from instructions for how to do things, to guidance on what to buy.

While social engineering is “not our problem,” it becomes our burden when we expect people to (gestures wildly with hands) have to independently think too much. It’s a delicate dance, how to bridge across what we all loathe, to how to keep our integrity—and our sustainability, as a largely volunteer led project.