Yeah, I understand. This is also the rationale for limiting template access in the first place.
Well, as for the PoC, that’s really @marmarek’s call.
Yeah, I understand. This is also the rationale for limiting template access in the first place.
Well, as for the PoC, that’s really @marmarek’s call.
And here we go the path of redoing guides to install specific softwares. Sigh.
Sorry. Didn’t want to upset you. In case of signal I too think that we should probably just add the repository to the debian-11 template. Not as a general solution, but recognizing that Qubes OS users are in fact highly likely to be signal users too. On the other hand this one concession would probably open the flood gates for all kinds of requests …
Upstream instructions are not unified! Without unified solution, it means
The general idea behind my proposition is based on accepted trust of upstream installation instructions (not to be confounded with commands given on other sources).
And then we open the gates of the old persona problem which is to have different templates downloadable/installable at OS install to fit different use cases with those repositories/softwares already installed. I’m not against the idea. But the current proposition was aimed at lowering the bar for users just wanting to follow as cose as possible upstream instructions.
Reminder. Signal instructions could simply be:
Users go on, says ok, ok, I can do this and then…
wget-proxy -O- https://updates.signal.org/desktop/apt/keys.asc | gpg --dearmor > signal-desktop-keyring.gpg
cat signal-desktop-keyring.gpg | sudo tee -a /usr/share/keyrings/signal-desktop-keyring.gpg > /dev/null
echo 'deb [arch=amd64 signed-by=/usr/share/keyrings/signal-desktop-keyring.gpg] https://updates.signal.org/desktop/apt xenial main' |\
sudo tee -a /etc/apt/sources.list.d/signal-xenial.list
sudo apt update && sudo apt install signal-desktop
He waits. shutdowns the template. Assign Signal to qube. Run it and…
Wow. It just worked ™.
He loves Qubes. Doesn’t doubt of the OEM who prepared his laptop of having broke the internet… He feels confident about life. And goes back at using applications, knowing it will be updated as any other.
(while we limited the possible impact of having followed shitty instructions downloading stuff to be autoran in all used qubes by following randomly found instructions, or having access to internet for anything else then wget and curl through wrappers using the proxy).
Bonus: he can generalize the concept instead of searching for answers, and then type obscure commands that were trusted because given by @sven on a random post on the forum. Who is @Sven ? Should he be trusted? I feel we are going in the opposite direction. Don’t get me wrong, I reviewed your commands and they are fine.
But… The absolute thing we do not want is users typing randomly found commands in forums with a template having full internet access!!!
The goal desired is journalists/whisleblowers not being tekksavvy being able to get home to commands translation, hand on on their shitty internet connection over tor and be able to install needed software without any complication, without opening holes in their templates, without them having to understand all the architecture, without them having to know qvm-run (that is teksavvy btw!!!) and just… seriously, just, be able to trust the open source software supply chain, understand rudiments of GPG and have just works ™ without looseing up security of their system without having to worry and trust who they should trust! You use Signal? Trust their instructions. Not a random post on the internet. That would be my ideal.
Now, the user will memorize weird process of replacing wget
(what?) with wget-proxy
(what? stupid Qubes, making complicated things even more complicated). Will it help security? I don’t think so. Instead, explicitly asking a user to create a dedicated non-trusted template should make them think about compartmentalization.
Indeed, it makes the cloned template less secure. But it unifies the installation experience, provides an easy onboarding to Qubes and opens a path to further secure the OS later. Remember, Qubes is a reasonably secure OS. This is how I understand it.
In my opinion, your suggestion is relevant for advanced uesrs and not for newbies. The former can already follow @Sven’s instructions (and verify them themselves), while the latter will simply be turned away by unnecessary complexity, which Qubes forces on them. Qubes is complex enough already without it.
I don’t think it’s a right place for this script, but it looks like an extremely helpful thing to me. If you could share your other scripts like this in a dedicated thread, you would help a lot of semi-advanced users, including me.
This is not necessarily true. I personally hate Signal for it’s walled-garden policy and for forcing users into Android or iOS. Please do not make a special case for it in Qubes.
@Sven your solution is good.
But…
Script requires to be copied over to dom0 (which needs complicated qvm-run commands from dom0 to have written copy-pasted script passed over qube to dom0), which is also not so desirable and implies running code on dom0.
This is why I reacted so much.
I am dozing out of my suggestion, really do not like the idea of giving internet access to even untrusted templates, but I agree with you: this might be to contain user errors who needs security the most, while I agree with @fsflover that wget-proxy would require from the user to convert upstream instructions everytime it fails(I liked that error/learning approach).
Where giving internet access to Template may be the easiest solution and should maybe be simply better documented in the core (@adw ) documentation, as Qubes documentation (which is the referecne for all users) was not current.
Edit: was done by @adw : awesome.
Meanwhile, I will simply deploy wget-proxy and curl-proxy to limit the flood in my support box. Ping me again if the idea of a better, upstreamed solution to deal with daily, real-users facing problems of wanting to add a trusted repository should be done in a non-duplicated, untrusted template (which duplicates network bandwith as well for updates, still today, and is not a luxury all end-users have).
@Insurgo I am not trying to be dense and I admit I haven’t read the entire thread – so ignore me if that’s over the line.
But now I have the impression what we are really talking about is to create Qubes OS specific scripts called ‘curl-proxy’ and ‘wget-proxy’ and deploy them to all templates so that we can then document somewhere:
If you see ‘curl’ or ‘wget’ in an install script, replace them with ‘curl-proxy’ and ‘wget-proxy’
… so you, I and other can in future just point to that documentation page and say “do this”.
But then, we could simply document:
In case of ‘curl’ add the ‘–proxy http://127.0.0.1:8082/’ parameter and in case of ‘wget’ add ‘-e use_proxy=yes -e http_proxy=127.0.0.1:8082’
Same effect, no changes in Qubes OS required. Alternatively we could educate users about ~/.wgetrc and
~/.curlrc and provide examples that already include the required proxy config. Yes?
In the case of Signal, Element, and other widely used applications, I plan to include the public keys in a package that is installed by default. This means that the “download the public key” step is not only unnecessary, it’s actively harmful: the copy in the OS is trustworthy, while the one downloaded might not be (if someone fooled a CA into giving them a rogue certificate, say).
More generally, I think @adw is much better than I am explaining these things than I am . This post of his is right on the money.
I see your point @demi. However, I don’t think in this particular case it will help the situation. Users won’t look for how to specifically install software X in Qubes. Rather they will look for upstream instructions (e.g. from signal’s website). So they will inevitable try to run the wget command to obtain the gpg. They’ll do all sort of things in a desperate attempt to get it to work.
In your solution this requires prior knowledge of what a GPG key is and that it’s available in the keyrings. But most new users will not know that. However, for more advanced users equiped with that knowledge it will be a nice to have.
On the other hand in @Insurgo’s solution the user will type the commands and see an error indicating a clear solution (replacing wget
with wget-proxy
) – something which hands the solution to the user instead of waiting for them to go look for it.
Here comes the PoC.
alias curl="curl-wrapper"
alias wget="wget-wrapper"
# sudo will not use aliases unless https://linuxhandbook.com/run-alias-as-sudo/
alias sudo="sudo "
#/etc/profile.d scripts are not loaded in interactive non-login shells (https://linuxhandbook.com/run-alias-as-sudo/)
if [ -d /etc/profile.d ]; then
for i in /etc/profile.d/*.sh; do
if [ -r $i ]; then
. $i
fi
done
unset i
fi
#!/bin/env bash
# wget-wrapper script around wget
# Goal is to give a chance to the user of knowing what is going to happen next, educate and explain what is different in Templates
#
# This wrapper script detects if the wget command was ran in a TemplateVM (Which by default is prohibited)
# In case the Template is trusted (No Network assigned), it warns the user to call wget-proxy instead and exits
# In case the Template has network assigned, it warns the user the template should not be trusted anymore and proceeds in 3 seconds
# In case we are not in a template, just add a warning and proceed with original wget call
#
#Depends on :
# this file being deployed in OS searchable path (eg: /usr/bin)
# alias being previously defined on the system prior of running wget.
# To test this:
# alias wget="wget-wrapper"
# To permanently define the alias for a user, add the alias in ~/.bashrc
if [ "$(qubesdb-read /type)" == "TemplateVM" ]; then
if ! $(qubesdb-read /qubes-gateway > /dev/null 2>&1); then
echo "You attempted to download a file with wget in a Template without direct internet access. (no network assigned)." >&2
echo "" >&2
echo "The attempted downloading command was: wget $@" >&2
echo "" >&2
echo "You either:" >&2
echo " 1- need to replace wget by wget-proxy in previous attempted shell command line (Recommended)." >&2
echo " 2- assign a Networking qube to this Template which should consequently be less trusted. (Untrusted)." >&2
exit 1
else
echo "You attempted to download a file with wget in a Template WITH DIRECT INTERNET ACCESS. (network assigned)." >&2
echo "" >&2
echo "The attempted downloading command was: wget $@" >&2
echo "" >&2
echo "Continuing in 3 seconds... Type CTRL+C to prevent the download now!" >&2
sleep 3
/usr/bin/wget "$@"
fi
else
echo "wget download attempt not in TemplateVM. Continuing..." >&2
/usr/bin/wget "$@"
fi
#!/bin/env bash
# curl-wrapper script around curl
# Goal is to give a chance to the user of knowing what is going to happen next, educate and explain what is different in Templates
#
# This wrapper script detects if the curl command was ran in a TemplateVM (Which by default is prohibited)
# In case the Template is trusted (No Network assigned), it warns the user to call curl-proxy instead and exits
# In case the Template has network assigned, it warns the user the template should not be trusted anymore and proceeds in 3 seconds
# In case we are not in a template, just add a warning and proceed with original curl call
#
#Depends on :
# this file being deployed in OS searchable path (eg: /usr/bin)
# alias being previously defined on the system prior of running curl.
# To test this:
# alias curl="curl-wrapper"
# To permanently define the alias for a user, add the alias in ~/.bashrc
if [ "$(qubesdb-read /type)" == "TemplateVM" ]; then
if ! $(qubesdb-read /qubes-gateway > /dev/null 2>&1); then
echo "You attempted to download a file with curl in a Template without direct internet access. (no network assigned)." >&2
echo "" >&2
echo "The attempted downloading command was: curl $@" >&2
echo "" >&2
echo "You either:" >&2
echo " 1- need to replace curl by curl-proxy in previous attempted shell command line (Recommended)." >&2
echo " 2- assign a Networking qube to this Template which should consequently be less trusted. (Untrusted)." >&2
exit 1
else
echo "You attempted to download a file with curl in a Template WITH DIRECT INTERNET ACCESS. (network assigned)." >&2
echo "" >&2
echo "The attempted downloading command was: curl $@" >&2
echo "" >&2
echo "Continuing in 3 seconds... Type CTRL+C to prevent the download now!" >&2
sleep 3
/usr/bin/curl "$@"
fi
else
echo "curl download attempt not in TemplateVM. Continuing..." >&2
/usr/bin/curl "$@"
fi
#!/bin/env bash
curl --proxy http://127.0.0.1:8082/ --tlsv1.2 --proto =https --max-time 180 $@
#!/bin/env bash
https_proxy=http://127.0.0.1:8082/ http_proxy=http://127.0.0.1:8082/ wget --secure-protocol=TLSv1_2 --timeout=180 "$@"
user@debian-11:~$ alias
alias curl='curl-wrapper'
alias ls='ls --color=auto'
alias sudo='sudo '
alias wget='wget-wrapper'
user@debian-11:~$ wget -O- https://updates.signal.org/desktop/apt/keys.asc
You attempted to download a file with wget in a Template without direct internet access. (no network assigned).
The attempted downloading command was: wget -O- https://updates.signal.org/desktop/apt/keys.asc
You either:
1- need to replace wget by wget-proxy in previous attempted shell command line (Recommended).
2- assign a Networking qube to this Template which should consequently be less trusted. (Untrusted).
user@debian-11:~$ wget-proxy -O- https://updates.signal.org/desktop/apt/keys.asc | gpg --dearmor > signal-desktop-keyring.gpg
--2022-04-21 10:52:38-- https://updates.signal.org/desktop/apt/keys.asc
Connecting to 127.0.0.1:8082... connected.
Proxy request sent, awaiting response... 200 OK
Length: 3090 (3.0K) [application/pgp-signature]
Saving to: ‘STDOUT’
- 100%[=======================================================================>] 3.02K 18.1KB/s in 0.2s
2022-04-21 10:52:41 (18.1 KB/s) - written to stdout [3090/3090]
user@debian-11:~$ alias
alias curl='curl-wrapper'
alias ls='ls --color=auto'
alias sudo='sudo '
alias wget='wget-wrapper'
user@debian-11:~$ sudo curl -so /etc/apt/trusted.gpg.d/oxen.gpg https://deb.oxen.io/pub.gpg
You attempted to download a file with curl in a Template without direct internet access. (no network assigned).
The attempted downloading command was: curl /etc/apt/trusted.gpg.d/oxen.gpg https://deb.oxen.io/pub.gpg
You either:
1- need to replace curl by curl-proxy in previous attempted shell command line (Recommended).
2- assign a Networking qube to this Template which should consequently be less trusted. (Untrusted).
user@debian-11:~$ sudo curl-proxy -so /etc/apt/trusted.gpg.d/oxen.gpg https://deb.oxen.io/pub.gpg
user@debian-11:~$ ls -al /etc/apt/trusted.gpg.d/oxen.gpg
-rw-r--r-- 1 root root 2213 Apr 21 11:58 /etc/apt/trusted.gpg.d/oxen.gpg
user@debian-11-networked:~$ wget -O- https://updates.signal.org/desktop/apt/keys.asc | gpg --dearmor > signal-desktop-keyring.gpg
You attempted to download a file with wget in a Template WITH DIRECT INTERNET ACCESS. (network assigned).
The attempted downloading command was: wget -O- https://updates.signal.org/desktop/apt/keys.asc
Continuing in 3 seconds... Type CTRL+C to prevent the download now!
--2022-04-21 12:01:23-- https://updates.signal.org/desktop/apt/keys.asc
Resolving updates.signal.org (updates.signal.org)... 172.64.155.131, 104.18.32.125, 2606:4700:4400::6812:207d, ...
Connecting to updates.signal.org (updates.signal.org)|172.64.155.131|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3090 (3.0K) [application/pgp-signature]
Saving to: ‘STDOUT’
- 100%[===================>] 3.02K --.-KB/s in 0s
2022-04-21 12:01:23 (13.0 MB/s) - written to stdout [3090/3090]
user@debian-11-networked:~$ sudo curl -so /etc/apt/trusted.gpg.d/oxen.gpg https://deb.oxen.io/pub.gpg
You attempted to download a file with curl in a Template WITH DIRECT INTERNET ACCESS. (network assigned).
The attempted downloading command was: curl -so /etc/apt/trusted.gpg.d/oxen.gpg https://deb.oxen.io/pub.gpg
Continuing in 3 seconds... Type CTRL+C to prevent the download now!
user@debian-11-networked:~$ ls -al /etc/apt/trusted.gpg.d/oxen.gpg
-rw-r--r-- 1 root root 2213 Apr 21 12:02 /etc/apt/trusted.gpg.d/oxen.gpg
user@debian-11-networked:~$ wget
You attempted to download a file with wget in a Template WITH DIRECT INTERNET ACCESS. (network assigned).
The attempted downloading command was: wget
Continuing in 3 seconds... Type CTRL+C to prevent the download now!
wget: missing URL
Usage: wget [OPTION]... [URL]...
Try `wget --help' for more options.
user@debian-11-networked:~$ curl
You attempted to download a file with curl in a Template WITH DIRECT INTERNET ACCESS. (network assigned).
The attempted downloading command was: curl
Continuing in 3 seconds... Type CTRL+C to prevent the download now!
curl: try 'curl --help' or 'curl --manual' for more information
user@wrapper-test:~$ wget
wget download attempt not in TemplateVM. Continuing...
wget: missing URL
Usage: wget [OPTION]... [URL]...
Try `wget --help' for more options.
user@wrapper-test:~$ curl
curl download attempt not in TemplateVM. Continuing...
curl: try 'curl --help' or 'curl --manual' for more information
@marmarek @adw @Sven @fsflover @Demi @deeplow : Am I making my point?
@Demi @marmarek: Fell upon two interesting bugs
alias sudo="sudo "
per How to Run an Alias With Sudo in Linux
This is why /etc/bash.bashrc
was modified to also source /etc/profile.d/*.sh
.
@marmarek @demi @fepitre : Qubes Bug/Feature?
Edit: Sorry for all the edits. I’m done now.
Edit2. I’m done for real, sorry. This is why writing documentation is so hard.
As you can see in my PoC
I hope that with the console output given in previous PoC post, I was able to prove my point and the pertinence of this massive UX improvement for Qubes OS (@marmata?). Of course, take it, modify it, improve it. But please consider the need and effectiveness of such solution.
Otherwise, as @deeplow said and I cannot agree more:
Missing piece:
An alternative idea to the alias: set https_proxy=http://127.0.0.1:8082
env variable. This will make curl/wget work automatically, in this terminal only. No need to change anything in upstream installation instruction then. The steps are then:
export https_proxy=http://127.0.0.1:8082
The command is easy to enter wrong, but we could have an alias like enable-network-proxy
. I think that’s significantly simpler than wrapping curl, wget and whatnot. And also, should allow only https traffic, but not plain http (at least in theory).
But, if users have to enter this, how will they know they need to enter this?
I was curious to trying out the various methods mentioned above so I made a deb-signal template only to find out none seem to work:
wget:
$ export https_proxy=http://127.0.0.1:8082
$ wget -O- https://updates.signal.org/desktop/apt/keys.asc
Connecting to 127.0.0.1:8082... connected.
Proxy tunneling failed: CONNECT denied (ask the admin to allow HTTPS tunnels)Unable to establish SSL connection.
curl:
$ curl --proxy http://127.0.0.1:8082/ --tlsv1.2 --proto =https --max-time 180 "https://updates.signal.org/desktop/apt/keys.asc"
curl: (56) Received HTTP code 403 from proxy after CONNECT
I then noticed that when launching the command, my apt-cacher qube was started and I found the following command to be working:
curl --proxy http://127.0.0.1:8082/ --tlsv1.2 --max-time 180 "http://HTTPS///updates.signal.org/desktop/apt/keys.asc"
So I had to remove --proto =https
and change https://
to http://HTTPS///
.
Is this behavior to be expected when not dealing with repositories but an apt-cacher qube is enabled?
It’s offtopic asking that here.
@adw @marmarek @sven @Demi @fsflover
If you prefer not interrupting user, but warning him that he is about to download stuff on the internet from a non-networked Template, that section could be changed to simply call the -proxy
(here curl-proxy as an exemple) instead of exiting and asking the user to change curl to curl-proxy manually…:
if [ "$(qubesdb-read /type)" == "TemplateVM" ]; then
if ! $(qubesdb-read /qubes-gateway > /dev/null 2>&1); then
echo "You attempted to download a file with curl in a Template without direct internet access. (no network assigned)." >&2
echo "" >&2
echo "The attempted downloading command was: curl $@." > &2
echo "" >&2
echo "Automatically translating to: curl-proxy $@" >&2
echo "" >&2
echo "Continuing in 3 seconds... Type CTRL+C to prevent the download now!" >&2
sleep 3
/usr/bin/curl-proxy "$@"
else
echo "You attempted to download a file with curl in a Template WITH DIRECT INTERNET ACCESS. (network assigned)." >&2
echo "" >&2
echo "The attempted downloading command was: curl $@" >&2
echo "" >&2
echo "Continuing in 3 seconds... Type CTRL+C to prevent the download now!" >&2
sleep 3
/usr/bin/curl "$@"
fi
else
echo "curl download attempt not in TemplateVM. Continuing..." >&2
/usr/bin/curl "$@"
fi
The resulting behavior would become similar to a Template with network access configured externally in Qubes, while using the proxy through curl-proxy translated call automatically (snippet):
The attempted downloading command was: curl -so /etc/apt/trusted.gpg.d/oxen.gpg https://deb.oxen.io/pub.gpg
Automatically translating to: curl-proxy -so /etc/apt/trusted.gpg.d/oxen.gpg https://deb.oxen.io/pub.gpg
Continuing in 3 seconds... Type CTRL+C to prevent the download now!
@deeplow @marmarek :
This seems like the only way to stop/warn/make aware/prevent users from running a whole snippet of code borrowed from the internet which would include curl/wget commands from actually doing harm and render otherwise trusted template into untrusted template, effectively limiting what would be downloaded before it happens.
The code snippet copy pasted in shell is going to download 3 files? The user receives 3 warnings! Other more obscure code snippets using something else then curl/wget will just not get access to the internet, which I think is the best tradeoff, not requiring to widen proxy access to the whole template nor giving global network access externally to the template.
Otherwise enticing the user to open the door wide open by either telling the whole template to use proxy / inviting the user to give internet globally to the Template per Qubes core documentation.
Yes, apt-cacher-ng is replacing the tinyproxy and listening in it’s place at 8082. You had to actually configure your apt-cacher in it’s config file to use that port.