Why are (most) Security Fixes Not Immediately Available?

The following is a discussion that stemmed from QSB-069: Multiple Xen and Intel issues.

In Qubes Security Bulletins like this one it is generally stated the following:

These packages will migrate from the security-testing repository to the
current (stable) repository over the next two weeks after being tested
by the community. [1] Once available, the packages are to be installed
via the Qubes Update Tool or its command-line equivalents. [2]

I will highlight the following quote:

Also, something just crossed my mind when trying to update as per recommendations here:

The Qubes Updater doesn’t provide an option to use the security-testing repo (or any other repo, for that matter), and use of the CLI command, sudo qubes-dom0-update, is being actively discouraged due to various issues. As far as I can tell after skimming the Salt documentation, users can’t choose their repo using this method either.

Any recommendations for nervous laypeople?

Also, how is one supposed to download, say, the Whonix template from the community templates repo without the CLI?

1 Like

Also, something just crossed my mind when trying to update as per recommendations here:

The Qubes Updater doesn’t provide an option to use the security-testing repo (or any other repo, for that matter), and use of the CLI command, sudo qubes-dom0-update, is being actively discouraged due to various issues. As far as I can tell after skimming the Salt documentation, users can’t choose their repo using this method either.

Any recommendations for nervous laypeople?

If you want to enable the security-testing repo, you can do this by
enabling the repository in the qubes-dom0.repo file in /etc/yum.repos.d
Salt calls should pick that up.

Also, how is one supposed to download, say, the Whonix template from the community templates repo without the CLI?

There is a template manager in the works which would allow users to
process and install templates - at the moment, the CLI is required
(whether directly or by using a salt formula)

1 Like

The path to the answer is already provided in the QSB:

These packages will migrate from the security-testing repository to the
current (stable) repository over the next two weeks after being tested
by the community. [1]

[...]

References
===========

[1] https://www.qubes-os.org/doc/testing/

However, the larger issue here is that the testing repos are for testers. When you enable any testing repo, you need to understand what you’re getting yourself into, including the risks and responsibilities you’re assuming.

Many regular users, especially non-technical folks, just want things to work and get upset and frustrated when they don’t. That is exactly the kind of user who should not be enabling any testing repos. The people who should be enabling testing repos are those who have the right expectations and mentality, are prepared to encounter problems, have the wherewithal to deal with them, and – most importantly – are willing to report the results of their testing to the Qubes developers in the correct way so that things can be fixed and improved. Remember, the whole point of testing any software is to find and fix bugs and improve things!

Over time, many of us have forgotten this and now see security-testing as merely a way to get security updates faster. The old QSB text, which provided the exact command required to get all updates immediately via security-testing, fed this misconception. People began to see each QSB as giving them a job saying, “In order to handle this QSB, enter this command in dom0.” But, when you pause to think about that, it makes no sense. If we truly wanted everyone to do this, there would be no need to tell everyone to manually enter a special command. We would just scrap security-testing and push all security updates immediately to the current branch so that updating normally would pick them up immediately.

In fact, we used to do this, but it turns out that, as with any software update, there can be bugs! And it turns out having testers test the updates before they go out to the entire userbase is a good way to catch and fix these bugs before they cause avoidable problems for everyone. Hence, security-testing was born. It exists for a reason!

Some of this was also discussed in QSB-068: Disconnecting a video output can cause XScreenSaver to crash.

No one ever said not to use the CLI for that. I think you’re misremembering the recommendation to “use Salt for updates” as “never use the CLI” or something. As the Updating Qubes OS page clearly states, “installing packages using direct package manager commands is fine,” and templates are installed as packages in dom0.

2 Likes

I just re-read my comment and noticed that I made a mistake in quoting XSA 374 when it’s not related to sys-net. I wasn’t thinking properly. I’ll correct this.

I understand where you’re coming from, but lots of people download and use the R4.1 alpha just for kicks and not for testing (especially non-technical folk like me who don’t know how to submit a proper bug report with the correct logs and references and posted in the right place). Some just feel more secure this way, regardless of whether this is actually the case. With QSB-related patches, things are slightly different, as there’s both identified vulnerabilities that are concerning enough to warrant security bulletins, and a fix for those now-disclosed vulnerabilities.

For a security-oriented OS, with security-conscious users (usually of the highest order: paranoid, or close to it), it would be downright cruel to dangle the fix and then say “This isn’t for you if you’re not really testing it”.This is true regardless of whether it will cause bugs, because fixing a known and disclosed vulnerability takes precedent over risk aversion to unintentional bugs, large or small.

I’m not saying this is true in my case (my computer isn’t involved in anything high-stakes), but this is my take on the issue of accessibility for laypersons who use Qubes.

 

My lack of knowledge is showing here, but aren’t updating and installing packages both dependent on similar (if not identical) mechanisms (e.g. RPM, dnf), and share the same security issues? Wouldn’t the issues that affect the update process be essentially replicated with the installation process? Apologies if this is something elementary.

1 Like

That’s a good example, because most users are probably less secure by switching from the stable release to an alpha. We do what we can to save users from shooting themselves in the feet like this, but there’s only so much we can do. For example, we can’t just not release things in development before they’re stable, because this is an open-source project, and we also need the testing feedback for development. At some point, baby-proofing the house makes the house unsuitable as a place to get work done. If we make a reasonable effort to educate users and steer them in the right direction, but some still insist on self-harm, where does it end? At some point, our time is just better spent elsewhere.

Good thing we’re not doing that, then! Instead, we’re saying, “The fix is in security-testing and will migrate to current in two weeks. Here’s where you can read more about that. Update normally, and you’ll get the fix when it’s available.”

A few important points here:

  1. Nothing is being “dangled,” because dangling implies showing you something that you can’t have. Anyone can get packages from security-testing (whether they should or not). If someone is too stupid to figure out how to do it by reading instructions written for them, they shouldn’t be trying it, and it would be a waste of time to try to make it easier for them, because that time would be better spent on things that actually matter and improve the world.
  2. There’s a huge difference between saying “if you want a gun, please follow this procedure” and “I know you never asked for one, but here’s a gun.” In neither case is the gun forbidden, yet the former is responsible, while the latter is insane.
  3. We’re explaining what the testing repos are intended for (namely, testing things). It’s up to users whether they choose to use them for their intended purpose. We don’t put any restrictions on them. We’re just declining to assist in their unintended (ab)use. Many of these complaints are essentially saying, “I want to use the testing repos, but I’m too lazy to take five minutes to read the documentation to learn how to enable them. Please spoon feed me the information instead.” My response is: “No. RTFM.”

You can’t make blanket statements like this and expect them to be true. It depends on many factors, such as one’s threat model, use case, risk tolerance, risk capacity, and so on. This statement is not true for many (probably even most) people. Many people value stability and reliability very highly, even if they also value security. Many people would rather be sure that they won’t lose important work in exchange for taking on some limited, temporary risk. (For example, many users would rather take the risk that their web browsing VM has a 50% chance of being compromised over the next week than a 50% chance that their hard drive would be wiped at some random moment over the next week.) There’s a reason that the standard practice for mainstream OSes is to delay security updates by at least a couple weeks before they reach all users. Many security experts consider this time period to be short enough that the risk is acceptable for mainstream users.

No, it’s not that the direct commands (e.g., dnf) are less secure now. Rather, it’s that Salt updates in Qubes can do extra security-enhancing stuff on top of them, and you’ll miss out on that if you only ever use the direct commands and don’t use Salt updates. So, it’s not that installing or updating with direct commands is insecure; it’s that Salt updates are more secure. There is no Salt method for installing stuff; only for updating. You have to use the direct commands for installing stuff, because there’s no other way (unless you just never install anything), so it makes no sense to object to this recommendation on the grounds that direct commands are still required for installing. That’s like saying, “I know that I can improve X without improving Y, but I’d rather just keep X and Y both equally bad.”

Here’s a draft post that explains more. Note that this is currently an un-reviewed, un-merged pull request. I don’t know whether the devs agree with it, so take it with a grain of salt. (PR #79)

https://github.com/QubesOS/qubes-posts/blob/update-commands/2021-05-06-updating-via-direct-commands-no-longer-recommended.md

2 Likes

Once the packages become available in the current (stable) repository, you’ll get the packages installed by updating your system as you usually do. :slightly_smiling_face:

2 Likes

While I agree that this is worth addressing, I think you might have misunderstood what I wrote earlier. I read the thread for the earlier QSB-068 and noticed a somewhat similar conversation played out there, so there might have been some confusion.

The key point I was making is intuitively it seems the risk of a bug dealing catastrophic, irreversible systemic damage is --from what I’ve observed-- lower than the risk of leaving a disclosed bug unpatched for two weeks. But this ultimately depends on users’ threat models.

I get what you’re saying: Stability is hugely important to the UX–losing all your data to a bug will turn some into rabid attack-dogs, and that’s something any developer would want to avoid. A community manager especially, since men-turned-raging-apes will fling excrement (women, too #genderequality), sometimes indiscriminately, and this just leads to a literal shitshow no community manager would voluntarily stick their head into.

From a UX standpoint, I get what you’re trying to avoid, but from a security standpoint, I feel the latter is less risky, all things considered. But, let’s say that a 10/10 super-critical bug in Xen was discovered and under attack in the wild–would you recommend a two-week wait while things got tested? What about 9.5/10? Where is the line drawn?

(I know CVE scores aren’t really useful, but you get the gist)

 

Not technically-trained; consume advice with salt

1 Like

That’s probably because your observation period coincides with security-testing existing and doing its job.

Also, your comparison equivocates on the word “risk.” In the phrase “the risk of a bug dealing catastrophic, irreversible systemic damage,” the word “risk” expresses the probability of “catastrophic, irreversible systemic damage” occurring. On the other hand, in the phrase “the risk of leaving a disclosed bug unpatched for two weeks,” the word risk attempts to express both the probability of and the magnitude of some negative outcome.

Someone might interpret this as an insinuation that I’m more concerned with making my own job easier than what’s best for users and for the project. I would assure such a person that this is not the case.

Oh, that one is easy. As already explained on the testing page, “Every new update is first uploaded to the security-testing repository if it is a security update or current-testing if it is a normal update. The update remains in security-testing or current-testing for a minimum of one week. On occasion, an exception is made for a particularly critical security update, which is immediately pushed to the current stable repository.” In other words, when the Qubes Security Team deems an update sufficiently critical, it bypasses security-testing.

What exactly is your point with all of this? Are you suggesting we discard industry-standard best practices and stop testing our security updates before shipping them (i.e., get rid of security-testing)? If not, what are you suggesting?

2 Likes

As would I–nobody in charge of a community wants to see it spiral into an angsty mosh-pit. Overall, having participated in Qubes forums for a year now, I think you’re doing a laudable job. I don’t have the time to properly respond to the rest right now, but I just wanted to make that explicit.

I don’t have a plan or a goal since at the beginning I just wanted to point out qubes-dom0-update isn’t favored but the Qubes Updater doesn’t have an obvious, clearly documented way of enabling security-testing or any other repo (for a newcomer, even a Linux veteran, the CLI is actually far more intuitive). The rest just followed naturally as part of the conversation. It’s good to know that " when the Qubes Security Team deems an update sufficiently critical, it bypasses security-testing", so thank you for clarifying that.

When I first started using Qubes, I literally went through every page on Documentation | Qubes OS, slowly and deliberately, over the course of a week, but there is still so much out there with Github and all, (and there’ll be much more too, with the qrexec overhaul), I just don’t have enough time or mental memory.

3 Likes

Thanks. :slight_smile:

Ok, that’s fine. I don’t mind the questions or discussion at all. I just wanted to make sure I got to the heart of the matter and clearly understood the specific actionable suggestion or request, if there was one.

Np.

That’s awesome! I wish there were more like you! :smiley:

I also don’t have the background to understand a lot of the developer documentation, but as a user of Qubes, it’s not required.

2 Likes