Curl-proxy / wget-proxy scripts in Templates so users can add GPG distro keys linked to added external repositories

@Demi Why curl-proxy and wget-proxy are not addressing the whole issue of downloading gpg distribution signing key per external documentation guides? That is what I wanted your input on. :slight_smile:

The non-technical users would still be confused when attempting to follow external software installation guides, which explicitly tells them to run specific commands in a specific order, which would still fail both their understanding and their final goal of having a software ready to use, receiving errors and not knowing those public GPG dist keys may/may not already be deployed. The end-user will still attempt to copy paste those commands, and those commands will still fail.

On that, @deeplow suggested to implement aliases, but as I understand his suggestion, it was to redirect wget/curl calls to their proxy wrapper counterparts, which I disagree. I kinda like the idea, though, but for another purpose altogether: inform the user of what he is attempting to do (download https://blahblah/dist.asc) and why it failed (direct access is prohibited unless explicitely typed), and tell him what to do instead (replace wget by wget-proxy in the previous command). We could have curl and wget aliased to non-functioning wrappers, telling the end user to call their wrapper alternatives instead if they really intended to access URL xyz?

As said there, implementing direct aliases to curl-proxy and wget-proxy would defeat the protection mechanism put in place by limiting internet access to update proxy itself. But having aliases pointing to curl-wrapper and wget-wrapper only outputting on screen what to use instead to access xyz seems to address concerns, reluctance and UX needed improvement.

Small research showed that tinyproxy configuration is not implementing any Logfile=/tmp/tinyproxy/proxy.log as of now. @marmarek : any reason for that? I think the update proxy should at least make that live log available to the end user from Qubes GUI somehow.

@adw @Demi @Sven? Challenges on the ideas of implementing curl-proxy/wget-proxy and curl and wget aliases to infor the user of what/why direct calls to wget and curl failed and tell them to use curl-proxy/wget-proxy instead (and having tinyproxy logs to inspect what happens)?

As a noob user who invests a lot of time learning Qubes, I find most useful when introducing Qubes to new users first thing after compartmentalization and isolation (even before that, because nowadays most of them are acustomed to virtualboxes and virtual machines) to explain them qrexec. Constantly, In a way Jehovah witnesses would do.
Once they realize that there is “some proxy” that saves them from the bad things from internet, whole system will be immensely user friendlier to them.

Qubes noob here

It seems to me that there’s an interesting evolutionary development occurring here:

  • Templates start out networked. It’s easy to install anything inside of them (and even browse the web!).
  • Developers, concerned about this, move to protect users from themselves by first limiting (via firewalling), then removing, network access.
  • Users get confused about how to install non-default-repo software.
  • People try to implement workarounds to make it easy to install anything inside of templates once again.

In a way, we have come full circle.

At the heart of the issue, I think we’re grappling with how to answer this question:

  • What is the point of removing network access from templates in order to protect users from themselves if we’re just going to give them ways to circumvent that very protection in ways they don’t understand?

And it has to be “in ways they don’t understand,” because if they understood, they wouldn’t need these easy workarounds. The whole point of drop-in *-proxy replacements is to allow users to blindly follow terminal commands they find somewhere on the internet. But if that’s what they’re doing, then the resultant templates should be untrusted. But if the resultant templates are untrusted, then what’s the point of removing network access from them in order to “protect” them? Once a user decides to make a template untrusted, there is no sense is trying to prevent the user from doing untrusted things inside of it.

10 Likes

Yes, maybe that a good idea. The user would run curl and get the error message:

Templates are offline so `wget` won't succeed. However, you can run `curl-proxy` to get files

I am sure there are users that don’t understand the processes, but are aware of the risks in general and do not blindly copy/pasting commands, that do not seek easy workarounds. I tried to speak for them too above.

Assertiveness might be the key word.

@adw Let’s go back to the idea of preventing network access to Templates for a minute. As per current deployed templates, user can install softwares that are included in defined and trusted repositories. Actual problem is to be able to deploy additional GPG public keys to verify signatures of packages for additional repositories. External trusted software repositories, with their own chain of trust and deployment signatures (authenticity, responsibility, integrity validation) is still the best and desired way of deploying softwares inside of Qubes OS.

Problem space

Qubes OS documentation, and contrib documentation, should not redo upstream software installation guides. Qubes OS should be able to provide a general solution, permitting to validate user expected (risky) choices, not permitting to do anything stupid in Templates, unless the user decides to shut down all the protections that were setuped to protect himself… against himself.

Proposed solution

  1. The proposed solution (wget-proxy and curl-proxy, if implemented as is), would prevent a user from being able to simply copy paste wget and curl commands coming from any source, since those commands do not have any proxy defined.
  2. The “alias” approach. Having aliases for wget pointing to wget-wrapper, and curl pointing to curl-wrapper, could intercept download attempts as @deeplow highlighted and explain to the user why that wget/curl call was intercepted, telling him to replace wget/curl with wget-proxy/curl-proxy. The wrapper could check if netvm is configured for the Template, and if not, warn the user of what he should do instead of calling those commands that cannot be used inside of a Template configured without network access for their protection.

Example

Combined, the end result could be, in the case of a user trying to install signal-desktop over debian-11:

  • wget -O- https://updates.signal.org/desktop/apt/keys.asc | gpg --dearmor > signal-desktop-keyring.gpg

    • wget cannot dowload https://updates.signal.org/desktop/apt/keys.asc since Template is configured to not have internet. If you really intended to download that file, please replace wget by wget-proxy in past command line.
  • wget-proxy -O- https://updates.signal.org/desktop/apt/keys.asc | gpg --dearmor > signal-desktop-keyring.gpg

Success.

Mitigation

  • What we want here is to prevent users to download public key in another VM, copy the public key over the Template, move things around where the user already is puzzled by those simple, relatively truste, instructions (keep Signal example in mind).
  • We want to prevent the user of bypassing the Template “no network” policy to follow standard external documentation.
  • We want the user to be able to add additional repositories (No @demi GPG public keys packaging won’t help the whole solution here of users still needing to add external repositories, and following upstream instructions is a different problem them trying to prevent users of doing completely crazy stuff in a template @adw. Hopefully they clone it prior anyway, and if they want to give network access to that template, then as you said, that template should not be trusted anymore.
  • We want the user to be able to learn what he does wrong and inform him of the security mechanisms in place. So if wget-wrapper and curl-wrapper are defined aliases, able to intercept wget and curl calls if there is no netvm, then telling him to replace his wget call by wget-proxy to download said file will make him understand what is happening, at least. And at best, will stop him from doing crazy stuff in a Template that has no netvm by default.

Makes sense?

@deeplow @adw:

This will still need a solution for users to download and install legitimately Element, Signal, Session and VPN related repositories to say the least. And I do not think that re-documenting each software installation guide to interpret upstream instructions is the way to go. A user intending to install Signal trusts Signal maintainers and their distribution chain. I do not think that complicating UX without explaining to the end-user what is actually happening differently inside of a Template and what he should do about it, on a more general approach, is a good long-term approach.

@adw: I am not saying that wget should be wget-proxy here. I am saying that a user running random scripts will most probably call wget and curl directly, which should fail, but that commands simply failing is not enough to understand (current) where calling wget-proxy as an alias for wget would be disastrous (random, malicious code snippets would be able to download whatever). I’m saying we should tell the user that if the intended file is really intended to be downloaded (wget-wrapper ouput) then the user should just replace wget with wget-proxy

Edit: change the URLs since this discussion has been moved to its own thread.

1 Like

@Insurgo I took the liberty to move this discussion into its own thread. Hope you don’t mind.

3 Likes

And posts will continue to flow. This one is from 3h ago :slight_smile:

@demi: I am not sure packaging public keys is the solution. Teaching and taking by the hand is, while global documentation (@adw: core) explaining how to add GPG keys should also be core, IMHO.

1 Like

@Demi

Tried real quick to try to do PoC wget-wrapper code snippet, and wanted insight on what is the best approach to get network info for Templates.

Without network

user@debian-11:~$ qubesdb-read /qubes-ip
Failed to read /qubes-ip
user@debian-11:~$ qubesdb-read /qubes-gateway
Failed to read /qubes-gateway
user@debian-11:~$ qubesdb-read /type
TemplateVM

With network:

user@debian-11-networked:~$ qubesdb-read /qubes-gateway
10.137.0.8
user@debian-11-networked:~$ qubesdb-read /qubes-ip
10.137.0.51
user@debian-11-networked:~$ qubesdb-read /type
TemplateVM

So basically:

  • wget could be an alias to wget-wrapper.
  • curl could be an alias to curl-wapper
  • wget-wrapper and curl-wrapper could check for qubesdb-read /qubes-gateway and qubesdb-read /type. If TemplateVM and have a gateway, warn the user that this is really insecure and ask for confirmation prior of continuing. If TemplateVM and no gateway, exit saying that -proxy countrpart should be used.

Thoughts?

Exactly this. The power of Qubes is that it does not change most workflows which people got used to – and still provides unbelievable security. I use it as if I used an ordinary Linux system (but with several independent enclaves).

I’m afraid that complications like offline templates may put people away. Adding fake curl makes things even more complicated in my opinion and goes against the policy of not changing the underlying operating systems. Please do not go this way. Same with GPG keys.

My suggestion is to allow as an option non-default, networked templates in order to install software outside the default repositories. Such option should be visible somewhere in the UI, e.g., a tick while creating a template. Also mention it in the documentation as an option for installing special software.

1 Like

I never get why getting functional ideas is so complicated.

  • Templates implement a proxy. It is only hidden, but used by OS software installer, which are modified to wrap around the proxy
  • The idea here is not to change the underlying operating system. But to wrap security measures and ease deployment.
  • Giving Templates a network gateway should be last resort, really.
  • Asking the user to download GPG Keys externally, qvm-copy them and then modify upstream installation instructions is a non-sensical and requires the users to be rtech-savvy, which is not the goal of Qubes, but a side effect of problems like those. We should lower friction (UX), without making it less secure.

[Looks like some of the ideas in this post were anticipated while I was typing it.]

Let’s step back for a moment. Why don’t templates have unrestricted network access anymore? To prevent users from doing dangerous things in templates, right? But this assumes that all templates are at the same ultra-high-trust security level, which isn’t realistic. In practice, many users want to have templates of varying trust levels. Some are ultimately trusted, some aren’t trusted very much at all, and others are in-between. Sometimes, users need to use some untrusted piece of software in a couple qubes, but they don’t trust it at all. They want to be able to install it in an untrusted template for use with untrusted app qubes in order to keep all of that separate from their trusted templates and app qubes. The problem is that the current one-size-fits-all policy of “no templates are allowed to have direct network access” doesn’t support this common use case. That’s what’s driving you to find workarounds.

See, that’s the thing. I’m not convinced that is the actual problem. That sounds more like a symptom of the underlying problem, which is the one-size-fits-all nature of the way template networking works. Suppose we added a feature where you could toggle on “unsafe mode” for a template, and doing so grants that template normal network access just like an app qube (and just like templates used to work back in the day). Then users would be able to add new keys to templates and install software in them the normal ways, without any proxy hoops to jump through. Of course, there should be a big red warning when enabling “unsafe mode” that explains just what you’re getting yourself into. And, if you prefer, you can instead have this “unsafe mode” (or whatever you’d like to call it) implement the networked-but-firewalled system we had more recently in Qubes history, where the user had to choose to allow temporary network access for 10-15 mins or however long.

Either way, we have the UX challenge of having to explain to the user why their desire to install (or enable the installation of) non-default-repo software is a security risk.

My point here isn’t that one solution would be better than the other, but rather that we should understand the nature of the underlying problem.

I can see the appeal of this approach, since the “direct network access” approach allows for doing “stupid” things in templates, but the trade-off is that the workaround is less robust. It’s only a matter of time until users start asking how to install software that doesn’t work with wget-proxy or curl-proxy. (Actually, they already are.)

Of course, this is just a specific instance of the more general trade-off between security and convenience.

Realistically, many (not all) users will follow upstream instructions without understanding them. Since those commands could do anything, this is tantamount to doing “crazy stuff” in a template. In other words, executing commands you don’t understand in a template is effectively doing crazy stuff in that template. But look, I’m not here to argue about whether it’s “crazy” or not. That’s beside the point. The point is that users want untrusted templates, but the current no-networking safeguard applies to all templates, hindering that use case.

This is not necessarily a mistake. After all, Qubes OS should be secure by default, so defaulting to the most secure assumptions for templates makes some sense. But that doesn’t change the reality that many users will continue to want untrusted templates and be stymied.

I agree with the user-education philosophy, but realistically, many users’ understanding will only get as far as “replacing wget with wget-proxy makes it work for some reason, so I just do that to make it work.”

Also, many users are allergic to the command line, so this is still a stepping stone to a GUI solution.

2 Likes

This is not about the default configuration. Default offline default templates are probably fine, although when I discovered them, I anticipated that many non-technical people will be annoyed by that.

I guess our disagreement comes from your expectation that Qubes should force extreme (relative of course) security, usability and freedom be damned. I disagree with that, and agree with @adw that a lower degree of security is a legitimate use case, because it’s a requirement for many users. Moreover, when someone switches to Qubes from a less secure systems, I don’t expect that they will immediately have maximally secure settings, but improve their security step by step. Starting from less secure templates might be a good starting point and is much better than any ordinary OS.

But changing curl utility does change the OS, doesn’t it?

First, thanks @adw for your thorough answer. On above quote, I do understand the GUI approach to install software already present in configured repositories. My whole idea challenge here is how to add additional repositories. Users allergic to terminal I can understand, but the GUI approach won’t resolve this same basic problem i’m trying to tackle here. Real customers recurrent problems!

And to me, that would be good enough. The user would need to do it manually, while wget-proxy could also give that same fat warning reminding the user that he is bypassing security measures on a trusted template.

I am not sure if its because I go into too much details that I do not make my point straight, reading @adw and @fsflover reactions to wrapper approach. This additional step would permit untrusted Templates to do whatever the user wants if we decide to (TemplateVM being detected by qubesdb-read /type, while networked state is detected by qubesdb-read /qubes-gateway).

What are other used methods (not curl/wget)?

Anyway, should I go forward with a PoC or this whole idea just doesn’t fit the Qubes OS vision (what is the plan for GUI GPG key import?)? Or am I not making my point clear enough?

I honestly don’t get what the problem is.

  1. users can create / clone as many templates as they like, deciding themselves which ones are trusted and which are untrusted

  2. the user can already assign a netvm to any template using the GUI should they wish to … no special checkbox or change required

  3. the user can already use curl --proxy http://127.0.0.1:8082/ in any template and it will work.

This is all available today and was since I joined this community in 2017 (R3.2). It’s documented in a hundred places. The only way to make this any easier are respective salt formula to be discovered, trusted and used by users.

1 Like

And my attempt was to address both use cases. There is nothing wrong with standalone qubes or insecure templates. But we are mixing things here. Step back: TemplateVMs are OS layer where nothing untrusted should be ran and shared amongst qubes having that base trust level in mind . A user can decide for himself to clone a Template qube and install whatever he wants there. No problem. But I do not see anything wrong into users wanting to deploy a new repository (including GPG public key…) into a trusted Template if that Template being debian-11 is a conscious choice and installing Signal repository there is ok. Nothing is ran by the deb package, nothing autostarts… Fits the bill. Going into other examples will give different results. But in that use case, like many others, the user should be able to download that GPG key without having to mess around.

The idea with the wrapper script would be to warn the user that what he is doing is actually to download something in the Template. If the user learns that calling wget-proxy actually downloads (bypass) no-network policy, but the big fat warning is still there (@fsflover: an alias is just a way to call the expected binary. You could set aliases of all sorts… alias ls="ls --color" being a shitty example, where alias curl=curl-wrapper would pass the command line from curl to curl-wrapper in all cases). So having a wrapper on dangerous commands, and modulate behavior depending of environement (qubesdb-read to the rescue) would fit both needs without too much complexity…)

  • .bashrc would append the aliases
  • curl-wrapper and wget-wrapper would contain the logic to deal specifically with TemplateVM type, and deal differently if there is gateway (network) or not.
  • curl-proxy and wget-proxy would not be called automatically, but the wrappers would invite users to adapt their commands when typed in TemplateVMs to confirm their understanding of the risks.

Otherwise, I have no other idea, folks. I just know that GUI installer will have to deal with this issue as well, and that most if not all customers have problems.

And no @Sven

As said in past comments, that would work without any change when upstream instructions are doing it ok, but if the call is wget -O- blablah, the user will be confused and annoyed and will ask for help… Once again, this is why secure wrappers would fit the bill, but seeing all this reluctance makes me doubt of the approach to resolve the real problem while people seem to mix things.

Non-technical users won’t know what to do! Right? We agree on that? No. Signal not being installable is not a good solution. And going with other software alternatives flatpak/snap/standalone is not a good approach because the user never knows if there are updates. Repositories, managed upstream, are still the best solution, IMOH. And installing GPG keys should not be a burden, untrusted/trusted templates not being a debate here. Something working for all cases should be possible… and what I was trying to achieve.

Oh, wow, you’re right! I never had a reason to try it myself. I just saw the pages of posts from confused users over the years and all the replies suggesting jumping through weird hoops, so naturally I assumed there must not be a trivially easy solution like simply… giving the template network access. There is no problem after all, then.

However, I also never encountered this in the official documentation. Seems like we should at least document this on the how to install software page. I just noticed that this section is out-of-date, since it still references changing only the template’s firewall rules, which of course won’t solve anything when the template has no NetVM.

1 Like

@Insurgo, what about this solution (using Signal as an example)?

User instructions:

  1. In the template’s settings, change Networking from default (none) (current) to sys-firewall.
  2. Open a terminal in that template.
  3. Follow the Signal website’s instructions.

Much simpler, no?

I’m just trying to get a concrete sense of where any remaining disagreements lie.

1 Like

Oh, I see you already addressed this a bit above.

But what’s the point of installing the Signal repository without using it to install the Signal app? The repository by itself does nothing unless it is used for installing packages, so there is no reason for it to exist in templates in which Signal is not installed.

…and then we’re back to just installing Signal in the template the normal way.

In this case,

Should become:

  • Clone your template first
  • In the new template’s settings, change Networking from default (none) (current) to sys-firewall
  • Go to firewall rules tab, select “Limit outgoing internet connection to…”
  • Tick “Allow full access for 5 minutes”
  • Open a terminal in that template.
  • Follow the Signal website’s instructions.

Mix/match 5 minutes with instructions to deactivate Template access when done. (But keep in mind that most users won’t turn it off, and that a lot of them then report weird behaviors from their qubes based on Templates they swear having be diligent but their bash history proves otherwise).

I suppose I had too many bad experiences with customers, which would go in all directions linked to a Template that was exposed to the internet without knowing the past exposure of such template. Users do errors. They will launch firefox that is assigned by default with the template (which I remove shortcut). They will type commands in Template that was aimed to be typed in qubes afterward. My background goes in the direction of limiting what can happen by default in a Template, pushing forward prevention mechanisms and making sure users understand what they are doing, but a lot do not understand. So limiting access to internet (as a big no) while permitting to convert generally simple instuctions (given by software providers) should be permitted with easy to understand swaps in commands. While still not permitting Firefox or any other installed software to be able to access the internet. But that is what I learned in the past years. I would love those users (which a lot of them are vulnerable and need those protections) to be safe by default, while not preventing them to actually do what they want to do.

If you have any improvements to suggest to push me into doing a PoC, let me know.

2 Likes