Curl-proxy / wget-proxy scripts in Templates so users can add GPG distro keys linked to added external repositories

General Discussion
1

Updates

Edit: Extended PoC of this idea is here. This is the blocking version (where non-networked TemplateVM will not translate curl/wget to curl-proxy/wget-proxy in the wrapper scripts.

Edit2: PoC code download here

Edit3: A non-blocking example for curl-wrapper automatically translates curl calls to curl-proxy calls under this post.

Edit4: No go. MOTD is a better approach. Proxy with static port is now considered risky by thread participants.

@deeplow @unman
I created issue Deploy curl-proxy/wget-proxy scripts so users can add GPG distro keys linked to added external repositories · Issue #7386 · QubesOS/qubes-issues · GitHub to propose a PoC that would permit users to manually call a curl-proxy script (which I think should be deployed in Qubes provided Templates by default), since it is currently impossible to download GPG keys from upstream package installation guides without the user having a clear understanding of all the commands implied, and where having such scripts (wget-proxy/curl-proxy) would permit users to benefit of the best way to maintain software packages under Qubes (which are: the best way to maintain OS related packages critical updates and normal package upgrade hygiene Qubes enforces by default).

As of now, the best way to install software under Qubes OS, shared between multiple qubes/proprietary softwares being seperated in a cloned specialized template, is still to install those additional software in the Templates themselves, where Qubes update widget will inform the user of available updates for those added reprositories as well, warning the user that updates are needed, and when applied, effective warnings are given to end users that they should restart their currently launched qubes to benefit from software upgrades that are now made available in templates. And where those warnings/GUI processed should receive maybe additional love.

The UX of installing new repositories under Qubes is limited by the capacity of users to be able to follow AND understand those upstream instructions without having to modify the installation guide given there, and the understanding of users of those commands and how to transform them into something that would work. Downloading gpg keys in disposable qubes, copying keys etc is too complicated for most of them.

curl-proxy

#!/bin/env bash
curl --proxy http://127.0.0.1:8082/ --tlsv1.2 --proto =https --max-time 180 "$@"

wget-proxy

#!/bin/env bash
https_proxy=http://127.0.0.1:8082/ http_proxy=http://127.0.0.1:8082/ wget --secure-protocol=TLSv1_2 --timeout=180 "$@"

Examples

Session-desktop documentation (https://deb.oxen.io/) drop in replacement:

sudo curl -so /etc/apt/trusted.gpg.d/oxen.gpg https://deb.oxen.io/pub.gpg
echo "deb https://deb.oxen.io $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/oxen.list
sudo apt update
sudo apt install session-desktop

Here the user would have to replace:
sudo curl -so /etc/apt/trusted.gpg.d/oxen.gpg https://deb.oxen.io/pub.gpg
to
sudo curl-proxy -so /etc/apt/trusted.gpg.d/oxen.gpg https://deb.oxen.io/pub.gpg

Signal-desktop instructions (Signal >> Download Signal for Linux) drop-in replacement:

Upstream instructions:

wget -O- https://updates.signal.org/desktop/apt/keys.asc | gpg --dearmor > signal-desktop-keyring.gpg
cat signal-desktop-keyring.gpg | sudo tee -a /usr/share/keyrings/signal-desktop-keyring.gpg > /dev/null
echo 'deb [arch=amd64 signed-by=/usr/share/keyrings/signal-desktop-keyring.gpg] https://updates.signal.org/desktop/apt xenial main' |\
  sudo tee -a /etc/apt/sources.list.d/signal-xenial.list
sudo apt update && sudo apt install signal-desktop

Here the following line needs to be changed:
wget -O- https://updates.signal.org/desktop/apt/keys.asc | gpg --dearmor > signal-desktop-keyring.gpg
becomes
wget-proxy -O- https://updates.signal.org/desktop/apt/keys.asc | gpg --dearmor > signal-desktop-keyring.gpg

I think that this would be a direct drop-in replacements for most frequently deployed additional repositories (please advise here) and should be pushed as official Qubes OS guidelines to be typed for software installation.

The curl-proxy/wget-proxy scripts examples force TLSV1.2 as a safer default, so the end user could simply replace wget and curl commands found in installation guides when he tries to install additional software and initial additional repositories instructions.

I would welcome implementation discussions to happen under the issue, where this comment in this thread is more to understand reluctance in applying this direct solution instead of pushing the user to use alternatives (flatpak/snap/appimages) which updates mechanisms are unclear for the user (am I running updated version? no. Its going to be maybe updated in the background and I will not be notified of updates needed).

7 Likes
2

One of my goals is to make signature verification easier to do, via a mechanism such as distribution-gpg-keys. That has been on my TODO list for quite a while.

2 Likes
3

@Demi Interesting, but that would mean packaging all external software verified GPG keys?

Please challenge me. Why providing wget-proxy and curl-proxy in qubes Templates would not fix the current issue? Most of the existing software providers provides at least debian/fedora packages. Users are mostly struggling with the GPG key part of those instructions since Templates do not have internet access. Having those, combined with a general documentation guideline “If wget/curl gives an error following upstream instructions, change wget/curl to wget-proxy/curl-proxy”. Done?

Of course, having minimalist GUI/Package manager which uses the Templae proxy to access the internet would ease life of GUI oriented people (survey will tell preference). But adding new repositories will most likely still require command line installation, unless all repositories for all available software are bundled inside of the Templates, which i’m not sure would be a good idea.

Thoughts?

4

Not all GPG keys, but many of them, yes. I suspect that including keys for some of the more popular packages would fix the problem for 90+% of users. Personally, I would be more inclined to use a StandaloneVM/flatpak/etc for the the remaining packages, to mitigate the consequences in the event the key turns out to be wrong.

2 Likes
5

@Insurgo I really quite like the elegance of your proposed solution. The GPG download step really confuses users. Making it seamless seems like the way to go.

Installing stuff in Linux is already way too messy for a non-technical user. Adding the complexity of understanding the commands’ meaning and knowing one has to get the GPG key from an internet-connected domain is just way too much complexity.

6

@Demi Why curl-proxy and wget-proxy are not addressing the whole issue of downloading gpg distribution signing key per external documentation guides? That is what I wanted your input on. :slight_smile:

The non-technical users would still be confused when attempting to follow external software installation guides, which explicitly tells them to run specific commands in a specific order, which would still fail both their understanding and their final goal of having a software ready to use, receiving errors and not knowing those public GPG dist keys may/may not already be deployed. The end-user will still attempt to copy paste those commands, and those commands will still fail.

On that, @deeplow suggested to implement aliases, but as I understand his suggestion, it was to redirect wget/curl calls to their proxy wrapper counterparts, which I disagree. I kinda like the idea, though, but for another purpose altogether: inform the user of what he is attempting to do (download https://blahblah/dist.asc) and why it failed (direct access is prohibited unless explicitely typed), and tell him what to do instead (replace wget by wget-proxy in the previous command). We could have curl and wget aliased to non-functioning wrappers, telling the end user to call their wrapper alternatives instead if they really intended to access URL xyz?

As said there, implementing direct aliases to curl-proxy and wget-proxy would defeat the protection mechanism put in place by limiting internet access to update proxy itself. But having aliases pointing to curl-wrapper and wget-wrapper only outputting on screen what to use instead to access xyz seems to address concerns, reluctance and UX needed improvement.

Small research showed that tinyproxy configuration is not implementing any Logfile=/tmp/tinyproxy/proxy.log as of now. @marmarek : any reason for that? I think the update proxy should at least make that live log available to the end user from Qubes GUI somehow.

@adw @Demi @Sven? Challenges on the ideas of implementing curl-proxy/wget-proxy and curl and wget aliases to infor the user of what/why direct calls to wget and curl failed and tell them to use curl-proxy/wget-proxy instead (and having tinyproxy logs to inspect what happens)?

7

As a noob user who invests a lot of time learning Qubes, I find most useful when introducing Qubes to new users first thing after compartmentalization and isolation (even before that, because nowadays most of them are acustomed to virtualboxes and virtual machines) to explain them qrexec. Constantly, In a way Jehovah witnesses would do.
Once they realize that there is “some proxy” that saves them from the bad things from internet, whole system will be immensely user friendlier to them.

Qubes noob here

8

It seems to me that there’s an interesting evolutionary development occurring here:

  • Templates start out networked. It’s easy to install anything inside of them (and even browse the web!).
  • Developers, concerned about this, move to protect users from themselves by first limiting (via firewalling), then removing, network access.
  • Users get confused about how to install non-default-repo software.
  • People try to implement workarounds to make it easy to install anything inside of templates once again.

In a way, we have come full circle.

At the heart of the issue, I think we’re grappling with how to answer this question:

  • What is the point of removing network access from templates in order to protect users from themselves if we’re just going to give them ways to circumvent that very protection in ways they don’t understand?

And it has to be “in ways they don’t understand,” because if they understood, they wouldn’t need these easy workarounds. The whole point of drop-in *-proxy replacements is to allow users to blindly follow terminal commands they find somewhere on the internet. But if that’s what they’re doing, then the resultant templates should be untrusted. But if the resultant templates are untrusted, then what’s the point of removing network access from them in order to “protect” them? Once a user decides to make a template untrusted, there is no sense is trying to prevent the user from doing untrusted things inside of it.

10 Likes
9

Yes, maybe that a good idea. The user would run curl and get the error message:

Templates are offline so `wget` won't succeed. However, you can run `curl-proxy` to get files
10

I am sure there are users that don’t understand the processes, but are aware of the risks in general and do not blindly copy/pasting commands, that do not seek easy workarounds. I tried to speak for them too above.

Assertiveness might be the key word.

11

@adw Let’s go back to the idea of preventing network access to Templates for a minute. As per current deployed templates, user can install softwares that are included in defined and trusted repositories. Actual problem is to be able to deploy additional GPG public keys to verify signatures of packages for additional repositories. External trusted software repositories, with their own chain of trust and deployment signatures (authenticity, responsibility, integrity validation) is still the best and desired way of deploying softwares inside of Qubes OS.

Problem space

Qubes OS documentation, and contrib documentation, should not redo upstream software installation guides. Qubes OS should be able to provide a general solution, permitting to validate user expected (risky) choices, not permitting to do anything stupid in Templates, unless the user decides to shut down all the protections that were setuped to protect himself… against himself.

Proposed solution

  1. The proposed solution (wget-proxy and curl-proxy, if implemented as is), would prevent a user from being able to simply copy paste wget and curl commands coming from any source, since those commands do not have any proxy defined.
  2. The “alias” approach. Having aliases for wget pointing to wget-wrapper, and curl pointing to curl-wrapper, could intercept download attempts as @deeplow highlighted and explain to the user why that wget/curl call was intercepted, telling him to replace wget/curl with wget-proxy/curl-proxy. The wrapper could check if netvm is configured for the Template, and if not, warn the user of what he should do instead of calling those commands that cannot be used inside of a Template configured without network access for their protection.

Example

Combined, the end result could be, in the case of a user trying to install signal-desktop over debian-11:

  • wget -O- https://updates.signal.org/desktop/apt/keys.asc | gpg --dearmor > signal-desktop-keyring.gpg

    • wget cannot dowload https://updates.signal.org/desktop/apt/keys.asc since Template is configured to not have internet. If you really intended to download that file, please replace wget by wget-proxy in past command line.

  • wget-proxy -O- https://updates.signal.org/desktop/apt/keys.asc | gpg --dearmor > signal-desktop-keyring.gpg

Success.

Mitigation

  • What we want here is to prevent users to download public key in another VM, copy the public key over the Template, move things around where the user already is puzzled by those simple, relatively truste, instructions (keep Signal example in mind).
  • We want to prevent the user of bypassing the Template “no network” policy to follow standard external documentation.
  • We want the user to be able to add additional repositories (No @demi GPG public keys packaging won’t help the whole solution here of users still needing to add external repositories, and following upstream instructions is a different problem them trying to prevent users of doing completely crazy stuff in a template @adw. Hopefully they clone it prior anyway, and if they want to give network access to that template, then as you said, that template should not be trusted anymore.
  • We want the user to be able to learn what he does wrong and inform him of the security mechanisms in place. So if wget-wrapper and curl-wrapper are defined aliases, able to intercept wget and curl calls if there is no netvm, then telling him to replace his wget call by wget-proxy to download said file will make him understand what is happening, at least. And at best, will stop him from doing crazy stuff in a Template that has no netvm by default.

Makes sense?

@deeplow @adw:

This will still need a solution for users to download and install legitimately Element, Signal, Session and VPN related repositories to say the least. And I do not think that re-documenting each software installation guide to interpret upstream instructions is the way to go. A user intending to install Signal trusts Signal maintainers and their distribution chain. I do not think that complicating UX without explaining to the end-user what is actually happening differently inside of a Template and what he should do about it, on a more general approach, is a good long-term approach.

@adw: I am not saying that wget should be wget-proxy here. I am saying that a user running random scripts will most probably call wget and curl directly, which should fail, but that commands simply failing is not enough to understand (current) where calling wget-proxy as an alias for wget would be disastrous (random, malicious code snippets would be able to download whatever). I’m saying we should tell the user that if the intended file is really intended to be downloaded (wget-wrapper ouput) then the user should just replace wget with wget-proxy

Edit: change the URLs since this discussion has been moved to its own thread.

1 Like
12

@Insurgo I took the liberty to move this discussion into its own thread. Hope you don’t mind.

3 Likes
13

And posts will continue to flow. This one is from 3h ago :slight_smile:

@demi: I am not sure packaging public keys is the solution. Teaching and taking by the hand is, while global documentation (@adw: core) explaining how to add GPG keys should also be core, IMHO.

1 Like
14

@Demi

Tried real quick to try to do PoC wget-wrapper code snippet, and wanted insight on what is the best approach to get network info for Templates.

Without network

user@debian-11:~$ qubesdb-read /qubes-ip
Failed to read /qubes-ip
user@debian-11:~$ qubesdb-read /qubes-gateway
Failed to read /qubes-gateway
user@debian-11:~$ qubesdb-read /type
TemplateVM

With network:

user@debian-11-networked:~$ qubesdb-read /qubes-gateway
10.137.0.8
user@debian-11-networked:~$ qubesdb-read /qubes-ip
10.137.0.51
user@debian-11-networked:~$ qubesdb-read /type
TemplateVM

So basically:

  • wget could be an alias to wget-wrapper.
  • curl could be an alias to curl-wapper
  • wget-wrapper and curl-wrapper could check for qubesdb-read /qubes-gateway and qubesdb-read /type. If TemplateVM and have a gateway, warn the user that this is really insecure and ask for confirmation prior of continuing. If TemplateVM and no gateway, exit saying that -proxy countrpart should be used.

Thoughts?

15

Exactly this. The power of Qubes is that it does not change most workflows which people got used to – and still provides unbelievable security. I use it as if I used an ordinary Linux system (but with several independent enclaves).

I’m afraid that complications like offline templates may put people away. Adding fake curl makes things even more complicated in my opinion and goes against the policy of not changing the underlying operating systems. Please do not go this way. Same with GPG keys.

My suggestion is to allow as an option non-default, networked templates in order to install software outside the default repositories. Such option should be visible somewhere in the UI, e.g., a tick while creating a template. Also mention it in the documentation as an option for installing special software.

1 Like
16

I never get why getting functional ideas is so complicated.

  • Templates implement a proxy. It is only hidden, but used by OS software installer, which are modified to wrap around the proxy
  • The idea here is not to change the underlying operating system. But to wrap security measures and ease deployment.
  • Giving Templates a network gateway should be last resort, really.
  • Asking the user to download GPG Keys externally, qvm-copy them and then modify upstream installation instructions is a non-sensical and requires the users to be rtech-savvy, which is not the goal of Qubes, but a side effect of problems like those. We should lower friction (UX), without making it less secure.
17

[Looks like some of the ideas in this post were anticipated while I was typing it.]

Let’s step back for a moment. Why don’t templates have unrestricted network access anymore? To prevent users from doing dangerous things in templates, right? But this assumes that all templates are at the same ultra-high-trust security level, which isn’t realistic. In practice, many users want to have templates of varying trust levels. Some are ultimately trusted, some aren’t trusted very much at all, and others are in-between. Sometimes, users need to use some untrusted piece of software in a couple qubes, but they don’t trust it at all. They want to be able to install it in an untrusted template for use with untrusted app qubes in order to keep all of that separate from their trusted templates and app qubes. The problem is that the current one-size-fits-all policy of “no templates are allowed to have direct network access” doesn’t support this common use case. That’s what’s driving you to find workarounds.

See, that’s the thing. I’m not convinced that is the actual problem. That sounds more like a symptom of the underlying problem, which is the one-size-fits-all nature of the way template networking works. Suppose we added a feature where you could toggle on “unsafe mode” for a template, and doing so grants that template normal network access just like an app qube (and just like templates used to work back in the day). Then users would be able to add new keys to templates and install software in them the normal ways, without any proxy hoops to jump through. Of course, there should be a big red warning when enabling “unsafe mode” that explains just what you’re getting yourself into. And, if you prefer, you can instead have this “unsafe mode” (or whatever you’d like to call it) implement the networked-but-firewalled system we had more recently in Qubes history, where the user had to choose to allow temporary network access for 10-15 mins or however long.

Either way, we have the UX challenge of having to explain to the user why their desire to install (or enable the installation of) non-default-repo software is a security risk.

My point here isn’t that one solution would be better than the other, but rather that we should understand the nature of the underlying problem.

I can see the appeal of this approach, since the “direct network access” approach allows for doing “stupid” things in templates, but the trade-off is that the workaround is less robust. It’s only a matter of time until users start asking how to install software that doesn’t work with wget-proxy or curl-proxy. (Actually, they already are.)

Of course, this is just a specific instance of the more general trade-off between security and convenience.

Realistically, many (not all) users will follow upstream instructions without understanding them. Since those commands could do anything, this is tantamount to doing “crazy stuff” in a template. In other words, executing commands you don’t understand in a template is effectively doing crazy stuff in that template. But look, I’m not here to argue about whether it’s “crazy” or not. That’s beside the point. The point is that users want untrusted templates, but the current no-networking safeguard applies to all templates, hindering that use case.

This is not necessarily a mistake. After all, Qubes OS should be secure by default, so defaulting to the most secure assumptions for templates makes some sense. But that doesn’t change the reality that many users will continue to want untrusted templates and be stymied.

I agree with the user-education philosophy, but realistically, many users’ understanding will only get as far as “replacing wget with wget-proxy makes it work for some reason, so I just do that to make it work.”

Also, many users are allergic to the command line, so this is still a stepping stone to a GUI solution.

2 Likes
18

This is not about the default configuration. Default offline default templates are probably fine, although when I discovered them, I anticipated that many non-technical people will be annoyed by that.

I guess our disagreement comes from your expectation that Qubes should force extreme (relative of course) security, usability and freedom be damned. I disagree with that, and agree with @adw that a lower degree of security is a legitimate use case, because it’s a requirement for many users. Moreover, when someone switches to Qubes from a less secure systems, I don’t expect that they will immediately have maximally secure settings, but improve their security step by step. Starting from less secure templates might be a good starting point and is much better than any ordinary OS.

But changing curl utility does change the OS, doesn’t it?

19

First, thanks @adw for your thorough answer. On above quote, I do understand the GUI approach to install software already present in configured repositories. My whole idea challenge here is how to add additional repositories. Users allergic to terminal I can understand, but the GUI approach won’t resolve this same basic problem i’m trying to tackle here. Real customers recurrent problems!

And to me, that would be good enough. The user would need to do it manually, while wget-proxy could also give that same fat warning reminding the user that he is bypassing security measures on a trusted template.

I am not sure if its because I go into too much details that I do not make my point straight, reading @adw and @fsflover reactions to wrapper approach. This additional step would permit untrusted Templates to do whatever the user wants if we decide to (TemplateVM being detected by qubesdb-read /type, while networked state is detected by qubesdb-read /qubes-gateway).

What are other used methods (not curl/wget)?

Anyway, should I go forward with a PoC or this whole idea just doesn’t fit the Qubes OS vision (what is the plan for GUI GPG key import?)? Or am I not making my point clear enough?

20

I honestly don’t get what the problem is.

  1. users can create / clone as many templates as they like, deciding themselves which ones are trusted and which are untrusted

  2. the user can already assign a netvm to any template using the GUI should they wish to … no special checkbox or change required

  3. the user can already use curl --proxy http://127.0.0.1:8082/ in any template and it will work.

This is all available today and was since I joined this community in 2017 (R3.2). It’s documented in a hundred places. The only way to make this any easier are respective salt formula to be discovered, trusted and used by users.

1 Like