[qubes-users] HCA reports - some advice please

hi I have got a new computer working, and it is a model new to Qubes
not just. (Guess who got it cheap on an early bird reduction on
kickstarter then :slight_smile:

So, I am going to send in the HCA report. I have produced the .yml
file. It contains some FIXME items. I am unclear: is it up to e to fix
them, or are they a note to whoever processes the report before
posting to the HCA page?

If I have to edit them, what do I use for the "short" items? Am I
reasonably free to abbreviate?

I am thinking of including the cpio files, but do not want to share a
serial number that they contain. WOuld those files be useful to others
if I edited them so that the serial number reads "Redacted"?

Finally, the manufacturer's name shown in the .yml is different from
the name they used on kickstarter. Would it help, or would it cause
confusion, if I added at the end of their name "t/a MinisForum"?

WArmly
R~~

hi I have got a new computer working, and it is a model new to Qubes
not just. (Guess who got it cheap on an early bird reduction on
kickstarter then :slight_smile:

So, I am going to send in the HCA report.

Thank you!

I have produced the .yml
file. It contains some FIXME items. I am unclear: is it up to e to fix
them, or are they a note to whoever processes the report before
posting to the HCA page?

If you can fix them first, that would be a great help! I think it would make things easier for our HCL maintainer. :slight_smile:

If I have to edit them, what do I use for the "short" items? Am I
reasonably free to abbreviate?

Usually, it's just the model number for that product, e.g., "FX-8320" is short for "AMD FX(tm)-8320 Eight-Core Processor". Take a look at the existing entries for examples:

I am thinking of including the cpio files, but do not want to share a
serial number that they contain. WOuld those files be useful to others
if I edited them so that the serial number reads "Redacted"?

Sure, feel free to redact whatever you like. :slight_smile:

If you prefer, you can send the cpio files directly to Marek PGP-encrypted (instead of the to the mailing list). See here for more info:

Finally, the manufacturer's name shown in the .yml is different from
the name they used on kickstarter. Would it help, or would it cause
confusion, if I added at the end of their name "t/a MinisForum"?

It would be ok, but perhaps including this in the comment field instead would make it more likely that your manufacturer field matches any other reports submitted for products from the same manufacturer.

I have a question about the HCL process and page display that I have been wondering about.

I was for the longest time copying and pasting the HCL web page into a spreadsheet just so I could sort and delete out all the old information, as I was looking to replace my desktop system with something more up to date. I can’t tell you how many times in the last three years I copied the HCL to this spreadsheet, and when my old desktop finally died I had to give up hope and just bought a new system sight unseen that was not on the list and I just hoped for the best. Fortunately, it worked out Ok.

As it is right now it is difficult and getting increasingly harder to find just the latest hardware on the list as it seems that by the time something appears on the list it is no longer even available for purchase. However, there are LOTS of machines that you could only find on eBay and many/most lack sufficient memory, BIOS, or current chipset support for the current Qubes R4.x system being developed. Old systems on the HCL are seemingly never updated, so you can’t tell which ones are still working and which ones have retired years ago. There are many items on that list even in the wrong categories (e.g. DIY System boards in the Desktop section when there is a separate section just for those) and I see no defined process by which to help change that.

My question is this: What would it take to get a set of simple filter options on that HCL webpage? Or, is there a way for someone to help clean up and better organize this list?

Going forward it is not all that helpful to see what was historically running, years ago, if they are no longer adequate for the current Qubes R4.x baseline. My inclination is this lists’ primary function should be to support those who are looking for some adequate hardware that could run the current baseline, and failing that test, it should be filtered out by default. Or maybe just filter by date added/updated?

Another thought is we should actively request those who successfully upgrade their systems to the latest baseline to resubmit their HCL thus showing that the same system is still capable of running the latest baseline number. I know matching old and new HCL reports would require some work, but I think if you want Qubes to be more popular this is a must.

At the very least the list should have a way to display only those currently running R4.x.x by default, but then let someone tweak the filter settings to look at older hardware if they choose to do so.

thanks,

Steve

If you can fix them first, that would be a great help! I think it would
make things easier for our HCL maintainer. :slight_smile:

Usually, it's just the model number for that product, e.g., "FX-8320" is
short for "AMD FX(tm)-8320 Eight-Core Processor". Take a look at the
existing entries for examples:

https://github.com/QubesOS/qubes-hcl/tree/master

I am thinking of including the cpio files, but do not want to share a
serial number that they contain. WOuld those files be useful to others
if I edited them so that the serial number reads "Redacted"?

Sure, feel free to redact whatever you like. :slight_smile:

If you prefer, you can send the cpio files directly to Marek
PGP-encrypted (instead of the to the mailing list). See here for more info:

https://www.qubes-os.org/doc/hcl/#generating-and-submitting-new-reports

I have a question about the HCL process and page display that I have been
wondering about.

I was for the longest time copying and pasting the HCL web page into a
spreadsheet just so I could sort and delete out all the old information, as
I was looking to replace my desktop system with something more up to date.
I can't tell you how many times in the last three years I copied the HCL to
this spreadsheet, and when my old desktop finally died I had to give up
hope and just bought a new system sight unseen that was not on the list and
I just hoped for the best. Fortunately, it worked out Ok.

As it is right now it is difficult and getting increasingly harder to find
just the latest hardware on the list as it seems that by the time something
appears on the list it is no longer even available for purchase.

Remember that these are almost all reports voluntarily submitted by users. If it's mostly old hardware, that's because few people with new hardware are submitting reports for that hardware. We can't force anyone to submit reports, and we usually can't get new hardware to generate reports on ourselves. Though, to be fair, the reports from the mailing list haven't been added in a while, so that might also be part of it.

However,
there are LOTS of machines that you could only find on eBay and many/most
lack sufficient memory, BIOS, or current chipset support for the current
Qubes R4.x system being developed. Old systems on the HCL are seemingly
never updated, so you can't tell which ones are still working and which
ones have retired years ago. There are many items on that list even in the
wrong categories (e.g. DIY System boards in the Desktop section when there
is a separate section just for those) and I see no defined process by which
to help change that.

My question is this: What would it take to get a set of simple filter
options on that HCL webpage?

This open issue is very similar to what you're asking:

I've just opened two PRs (linked to this issue) that make the HCL tables sortable. However, some rows break on sorting. Please see the issue comments for more details and an image showing exactly how it breaks. If you can help with this, please let me know on that issue.

Or, is there a way for someone to help clean
up and better organize this list?

There are two main ways you can help:

1. Help un-break the aforementioned sorting, or provide a better way to sort or filter the tables.

2. Submit a PR that modifies or removes old or bad HCL entries:

    https://github.com/QubesOS/qubes-hcl/tree/master

Going forward it is not all that helpful to see what was historically
running, years ago, if they are no longer adequate for the current Qubes
R4.x baseline. My inclination is this lists' primary function should be to
support those who are looking for some adequate hardware that could run the
current baseline, and failing that test, it should be filtered out by
default. Or maybe just filter by date added/updated?

I can understand the motivation behind removing old entries for EOL Qubes releases. If those entries are truly of no use to anyone, then there is not much reason to keep them around. But perhaps there's some value in keeping the old entries that we're overlooking. I'm curious whether Chris and Marek have any opinions on this.

Another idea is to have separate HCL tables for each Qubes release, or even entirely separate HCL *pages* for each Qubes release. This might make sense as part of our plan for release-specific documentation:

Another thought is we should actively request those who successfully
upgrade their systems to the latest baseline to resubmit their HCL thus
showing that the same system is still capable of running the latest
baseline number. I know matching old and new HCL reports would require some
work, but I think if you want Qubes to be more popular this is a must.

We can request it, but I'm not sure how much uptake we'll get. In practice, someone would probably have to volunteer to take on the task of making these requests. Alternatively, we could suggest in the HCL documentation something like, "If you're interested in a model, but the HCL report is out-of-date, try asking the reporter to update their report."

Theoretically, there could be an automated system that emails people to ask them to update their reports periodically (annually, for example), but again, someone would have to volunteer to help set up such a system.

At the very least the list should have a way to display only those
currently running R4.x.x by default, but then let someone tweak the filter
settings to look at older hardware if they choose to do so.

Would be good, but we need help with the code (see above).

I have a question about the HCL process and page display that I have been
wondering about.

I was for the longest time copying and pasting the HCL web page into a
spreadsheet just so I could sort and delete out all the old information, as
I was looking to replace my desktop system with something more up to date.
I can’t tell you how many times in the last three years I copied the HCL to
this spreadsheet, and when my old desktop finally died I had to give up
hope and just bought a new system sight unseen that was not on the list and
I just hoped for the best. Fortunately, it worked out Ok.

As it is right now it is difficult and getting increasingly harder to find
just the latest hardware on the list as it seems that by the time something
appears on the list it is no longer even available for purchase.

Remember that these are almost all reports voluntarily submitted by
users. If it’s mostly old hardware, that’s because few people with new
hardware are submitting reports for that hardware.

Agreed. But it is certainly possible to make this more of a discussion on how to give back to the community. The Qubes patriotic thing to do is to submit your successes so others can follow without so much fear and hesitation.

We can’t force anyone
to submit reports, and we usually can’t get new hardware to generate
reports on ourselves.

No, but a well-placed note/request at the end of the Qubes install process could go a long way to actually encourage them to submit the report to help others. The “how you can help” could also suggest this as a way to give back which is easy even for novices who were just introduced to Qubes. Make it a badge of honor. In fact, one could encourage people with questions to include a report link/ID where the fundamentals of their basic machine configuration would be available online for the experts to better understand the problem. Not everyone would necessarily want to give their anonymity away, but for some questions, this link could provide some valuable information about the hardware that would be easy to share.

Though, to be fair, the reports from the mailing
list haven’t been added in a while, so that might also be part of it.

Very true, unfortunately. I submitted my “Dell XPS 8930” but it has not shown up yet. With 8 cores and 64GB of memory, it is already out of production but it is still available through other retailers. Somebody who is looking for a new beefy desktop may not see this on the HCL until it is no longer available anywhere. That is the same boat I was in when my desktop up and died and I had no choice but to draw straws and pick one almost at random. Yes, there were other very old XPS’s on the HCL and some did not work properly, but based on the hardware in this one I figured it might just work. Unfortunately, this only has a “firmware TPM” that is disabled in BIOS when using the legacy boot settings and there is no header on the motherboard to even add a physical TPM. I may just dabble with the idea of a qubes auditable software-based vTPM (qTPM) and see if I can find a way to make something work for the contributor’s packages. Not sure about that yet, but it’s an idea that might even allow for locking down the boot partition by making it read-only until after a successful boot/login. Evil maids can’t change what they can’t edit.

However,
there are LOTS of machines that you could only find on eBay and many/most
lack sufficient memory, BIOS, or current chipset support for the current
Qubes R4.x system being developed. Old systems on the HCL are seemingly
never updated, so you can’t tell which ones are still working and which
ones have retired years ago. There are many items on that list even in the
wrong categories (e.g. DIY System boards in the Desktop section when there
is a separate section just for those) and I see no defined process by which
to help change that.

My question is this: What would it take to get a set of simple filter
options on that HCL webpage?

This open issue is very similar to what you’re asking:

https://github.com/QubesOS/qubes-issues/issues/3795

I’ve just opened two PRs (linked to this issue) that make the HCL tables
sortable. However, some rows break on sorting. Please see the issue
comments for more details and an image showing exactly how it breaks. If
you can help with this, please let me know on that issue.

Or, is there a way for someone to help clean
up and better organize this list?

There are two main ways you can help:

  1. Help un-break the aforementioned sorting, or provide a better way to
    sort or filter the tables.

  2. Submit a PR that modifies or removes old or bad HCL entries:

https://github.com/QubesOS/qubes-hcl/tree/master

Good news: I am a retired professional computer scientist and willing to help.
Bad news: I retired for medical reasons and am now spending most of my time starting up a 501©(3) nonprofit to research an incurable disease that doesn’t even have a test for it yet.

I’ll be happy to do what I am able, but at the moment I have no clue how the current HCL infrastructure works. If someone can kindly help me get oriented as to what currently happens maybe I can help come up with a better way of doing it. I feel strongly that this is important enough that it needs doing to advance the cause of Qubes with more potential users.

Going forward it is not all that helpful to see what was historically
running, years ago, if they are no longer adequate for the current Qubes
R4.x baseline. My inclination is this lists’ primary function should be to
support those who are looking for some adequate hardware that could run the
current baseline, and failing that test, it should be filtered out by
default. Or maybe just filter by date added/updated?

I can understand the motivation behind removing old entries for EOL
Qubes releases. If those entries are truly of no use to anyone, then
there is not much reason to keep them around. But perhaps there’s some
value in keeping the old entries that we’re overlooking. I’m curious
whether Chris and Marek have any opinions on this.

I would not want to just throw that information away, but rather have the user actively adjust the settings to be able to review the older historical entries if desired. I did see that the HCL repository has the original yml files, so I would think that creating a database or index of them to allow better filter options would be a good first step. But I guess I’ll have to read the part about the current broken sorting algorithm to figure out what is happening before suggesting too much about what to do.

Another idea is to have separate HCL tables for each Qubes release, or
even entirely separate HCL pages for each Qubes release. This might
make sense as part of our plan for release-specific documentation:

https://github.com/QubesOS/qubes-issues/issues/5308

Another thought is we should actively request those who successfully
upgrade their systems to the latest baseline to resubmit their HCL thus
showing that the same system is still capable of running the latest
baseline number. I know matching old and new HCL reports would require some
work, but I think if you want Qubes to be more popular this is a must.

We can request it, but I’m not sure how much uptake we’ll get. In
practice, someone would probably have to volunteer to take on the task
of making these requests. Alternatively, we could suggest in the HCL
documentation something like, “If you’re interested in a model, but the
HCL report is out-of-date, try asking the reporter to update their report.”

Theoretically, there could be an automated system that emails people to
ask them to update their reports periodically (annually, for example),
but again, someone would have to volunteer to help set up such a system.

At the very least the list should have a way to display only those
currently running R4.x.x by default, but then let someone tweak the filter
settings to look at older hardware if they choose to do so.

Would be good, but we need help with the code (see above).

I’m willing to take a look, but can’t promise how much time I can actually devote. My medical situation currently dictates everything at the moment.

thanks for the quick reply,

Steve

I have a question about the HCL process and page display that I have been
wondering about.

I was for the longest time copying and pasting the HCL web page into a
spreadsheet just so I could sort and delete out all the old information,

as

I was looking to replace my desktop system with something more up to

date.

I can't tell you how many times in the last three years I copied the HCL

to

this spreadsheet, and when my old desktop finally died I had to give up
hope and just bought a new system sight unseen that was not on the list

and

I just hoped for the best. Fortunately, it worked out Ok.

As it is right now it is difficult and getting increasingly harder to

find

just the latest hardware on the list as it seems that by the time

something

appears on the list it is no longer even available for purchase.

Remember that these are almost all reports voluntarily submitted by
users. If it's mostly old hardware, that's because few people with new
hardware are submitting reports for that hardware.

Agreed. But it is certainly possible to make this more of a discussion on
how to give back to the community. The Qubes patriotic thing to do is to
submit your successes so others can follow without so much fear and
hesitation.

We can't force anyone

to submit reports, and we usually can't get new hardware to generate
reports on ourselves.

No, but a well-placed note/request at the end of the Qubes install process
could go a long way to actually encourage them to submit the report to help
others. The "how you can help" could also suggest this as a way to give
back which is easy even for novices who were just introduced to Qubes. Make
it a badge of honor. In fact, one could encourage people with questions to
include a report link/ID where the fundamentals of their basic machine
configuration would be available online for the experts to better
understand the problem. Not everyone would necessarily want to give their
anonymity away, but for some questions, this link could provide some
valuable information about the hardware that would be easy to share.

Good ideas!

I've opened an issue for this:

I've already implemented the documentation suggestions (as well as some similar ones inspired by your suggestions). For details, see the issue.

Though, to be fair, the reports from the mailing
list haven't been added in a while, so that might also be part of it.

Very true, unfortunately. I submitted my "Dell XPS 8930" but it has not
shown up yet. With 8 cores and 64GB of memory, it is already out of
production but it is still available through other retailers. Somebody who
is looking for a new beefy desktop may not see this on the HCL until it is
no longer available anywhere. That is the same boat I was in when my
desktop up and died and I had no choice but to draw straws and pick one
almost at random. Yes, there were other *very old* XPS's on the HCL and
some did *not* work properly, but based on the hardware in this one I
figured it might just work. Unfortunately, this only has a "firmware TPM"
that is disabled in BIOS when using the legacy boot settings and there is
no header on the motherboard to even add a physical TPM. I may just dabble
with the idea of a qubes auditable software-based vTPM (qTPM) and see if I
can find a way to make something work for the contributor's packages. Not
sure about that yet, but it's an idea that might even allow for locking
down the boot partition by making it read-only until after a successful
boot/login. Evil maids can't change what they can't edit.

We've decided to recruit an additional HCL maintainer to get reports from the mailing list onto the website more quickly. I've sent a message to these lists about it.

However,

there are LOTS of machines that you could only find on eBay and many/most
lack sufficient memory, BIOS, or current chipset support for the current
Qubes R4.x system being developed. Old systems on the HCL are seemingly
never updated, so you can't tell which ones are still working and which
ones have retired years ago. There are many items on that list even in

the

wrong categories (e.g. DIY System boards in the Desktop section when

there

is a separate section just for those) and I see no defined process by

which

to help change that.

My question is this: What would it take to get a set of simple filter
options on that HCL webpage?

This open issue is very similar to what you're asking:

https://github.com/QubesOS/qubes-issues/issues/3795

I've just opened two PRs (linked to this issue) that make the HCL tables
sortable. However, some rows break on sorting. Please see the issue
comments for more details and an image showing exactly how it breaks. If
you can help with this, please let me know on that issue.

Or, is there a way for someone to help clean
up and better organize this list?

There are two main ways you can help:

1. Help un-break the aforementioned sorting, or provide a better way to
sort or filter the tables.

2. Submit a PR that modifies or removes old or bad HCL entries:

     https://github.com/QubesOS/qubes-hcl/tree/master

Good news: I am a retired professional computer scientist and willing to
help.
Bad news: I retired for medical reasons and am now spending most of my
time starting up a 501(c)(3) nonprofit to research an incurable disease
that doesn't even have a test for it yet.
I'll be happy to do what I am able, but at the moment I have no clue how
the current HCL infrastructure works. If someone can kindly help me get
oriented as to what currently happens maybe I can help come up with a
better way of doing it. I feel strongly that this is important enough that
it needs doing to advance the cause of Qubes with more potential users.

I'm very sorry to hear that. We'd be very grateful for whatever time you can spare.

Basically, the current system is that all the HCL .yml reports reside in this repo (which is a submodule of the main website repo):

These are displayed on the web page you see, the source code of which is here:

In order for reports to get into the HCL repo, Chris (tasket), our HCL maintainer, collects them from the mailing list, performs any necessary cleanup and processing, then submits them in a PR to the HCL repo, which I then review and merge.

Chris can provide further details, as he has the most experience in this role.

Going forward it is not all that helpful to see what was historically

running, years ago, if they are no longer adequate for the current Qubes
R4.x baseline. My inclination is this lists' primary function should be

to

support those who are looking for some adequate hardware that could run

the

current baseline, and failing that test, it should be filtered out by
default. Or maybe just filter by date added/updated?

I can understand the motivation behind removing old entries for EOL
Qubes releases. If those entries are truly of no use to anyone, then
there is not much reason to keep them around. But perhaps there's some
value in keeping the old entries that we're overlooking. I'm curious
whether Chris and Marek have any opinions on this.

I would not want to just throw that information away, but rather have the
user actively adjust the settings to be able to review the older historical
entries if desired. I did see that the HCL repository has the original yml
files, so I would think that creating a database or index of them to allow
better filter options would be a good first step. But I guess I'll have to
read the part about the current broken sorting algorithm to figure out what
is happening before suggesting too much about what to do.

Since they're managed in a git repo, we're never in danger of permanently losing the data. However, being accessible to website visitors is another matter.

Since it's just a directory full of .yml files, perhaps there's some database or indexing magic that can be done. That's outside of my area of expertise.

Another idea is to have separate HCL tables for each Qubes release, or

even entirely separate HCL *pages* for each Qubes release. This might
make sense as part of our plan for release-specific documentation:

https://github.com/QubesOS/qubes-issues/issues/5308

Another thought is we should actively request those who successfully
upgrade their systems to the latest baseline to resubmit their HCL thus
showing that the same system is still capable of running the latest
baseline number. I know matching old and new HCL reports would require

some

work, but I think if you want Qubes to be more popular this is a must.

We can request it, but I'm not sure how much uptake we'll get. In
practice, someone would probably have to volunteer to take on the task
of making these requests. Alternatively, we could suggest in the HCL
documentation something like, "If you're interested in a model, but the
HCL report is out-of-date, try asking the reporter to update their report."

Theoretically, there could be an automated system that emails people to
ask them to update their reports periodically (annually, for example),
but again, someone would have to volunteer to help set up such a system.

At the very least the list should have a way to display only those
currently running R4.x.x by default, but then let someone tweak the

filter

settings to look at older hardware if they choose to do so.

Would be good, but we need help with the code (see above).

I'm willing to take a look, but can't promise how much time I can actually
devote. My medical situation currently dictates everything at the moment.

I understand. Thank you.