You understand that the updateVM does not need to have dom0
repositories defined, so you must understand that dom0 repo definitions
are passed threw to the updateVM.
Why would you think that qvm-template would not follow the same pattern?
This is a puzzle to me, but a matter of psychology.
Let’s look at code.
I have told you that the updateVM does not need to have the template
repo definitions installed, and I have shown you how to prove this for
yourself.
In dom0, look at /usr/lib/python3.8/site-packages/qubesadmin/tools/qvm_template.py qrexec_payload builds the payload for the qrexec call to the updateVM.
You can see the payload write ---, and then pulls in the
repository definitions.
payload += '---\n'
for path in args.repo_files:
with open(path, 'r', encoding='utf-8') as fd:
payload += fd.read() + '\n'
This is then used for the qrexec call to the updateVM
On the updateVM, the qrexec function is /etc/qubes-rpc/qubes.TemplateDownload,
which calls /usr/lib/qubes/qvm-template-repo-query
There you will see:
while IFS= read -r line; do
if [ "$line" = "---" ]; then
break
fi
case "$line" in
--enablerepo=*|\
--disablerepo=*|\
--repoid=*|\
--releasever=*|\
--refresh)
OPTS+=("$line")
;;
*)
SPEC="$line"
;;
esac
done
repodir=$(mktemp -d)
trap 'rm -r "$repodir"' EXIT
cat > "$repodir/template.repo"
The incoming payload is parsed until the --- signal, then what follows
is written out to a temporary template.repo, used in the dnf call.
This is how the updateVM can use the template repository definitions from
dom0 without them being “installed”.
Again, this process works under latest updates in 4,1 for Debian and
Fedora based updateVM qubes.
Apparently, it does not work for you.
I hope this helps you to troubleshoot your problem and identify the root
of your issue.
As an aside, why are there Debian and Fedora packages qubes-repo-template, containing repository definitions for Qubes
templates?
You can see that it is a dependency of qubes-core-admin-client, a
package that provides “Tools to manage Qubes system using Admin API”
If you use an adminVM, you have to have the template repository
definitions there.
I think this is not your situation.
I never presume to speak for the Qubes team.
When I comment in the Forum or in the mailing lists I speak for myself.
@unman: you are totally right. I forgot to check on which depends qubes-repo-templates, which would have answered my question. Your analysis of code went deeper than my own and shows that qubes-dom0-updater and qvm-templates-gui are reusing a similar but different logic, that is, to copy the repository definitions from dom0 to its updatevm, but uses those repo list differently: one through the update proxy (qvm-template) the other without (qubes-dom0-update).
My setup, as written elsewhere, encompasses whonix support as of now by having apt-cacher-ng proxy report (lying) that apt-cacher-ng is torrified at all times (that is, by modifying its offered web page which is checked by whonix templates prior of permitting the proxy to be accessed).
My setup activates qubes update notifications to dom0 as per normal Qubes deployment (qvm-service sys-whonix qubes-update-check on). To be able to report updates to dom0, those qubes also need to be able to download packages information (qvm-service sys-whonix updates-proxy-setup on) through the update proxy, and to do so, needs to have proper modified and compatible apt-cacher-ng repository definitions. And for those qubes to be able to talk to cacher, dom0 needs to have @anyvm policy under 30-user.policy file: qubes.UpdatesProxy * @anyvm @default allow target=cacher.
You covered this elsewhere and for that i’m sorry:
Basically, as of now, it is not possible to have cacher reporting qubes’s Templates available updates to dom0 unless a downstream everyday used qube has both qubes-update-check and updates-proxy-setup services enabled. But by doing so, it is impossible to use qvm-template or qvm-template-gui with a dom0 updatevm configured to use cacher.
So long story short, since my dom0 is configured to have sys-whonix has its updatevm, or any other template also configured to use cacher, it is impossible to have both downstream qubes reporting for its parent template available updates AND be used to download templates through qvm-template/qvm-template-gui.
So, the fix on my setup was to deactivate updates-proxy-setup service on dom0’s defined updatevm:
[user@dom0 ~]$ qvm-service sys-whonix
qubes-update-check on
updates-proxy-setup off
Doing so, running qvm-template-gui/qvm-template succeeds. But of course, sys-whonix doesn’t report to dom0 availabe updates unless whonix-gw-16 Template is started by itself. It’s also impossible to download temporarily (session-based) packages under sys-whonix, since sys-whonix doesn’t have standard repository definitions, since whonix-gw-16 still has its repositories modified to use apt-cacher-ng (most common with debian-11).
As I explained in different posts, I really appreciate Qubes to be able to install software momentarily in app qubes. This permits to install software that won’t persist across reboots, for example in sys-net to do some forensic analysis. Same for my app dvms where I do not see advantage in having those in parent Templates. We had that discussion for which your conclusion was totally valid: users might be confused into why software they installed vanished on next reboot of that app qube. This is valid, but also permitted in normal Qubes installation and really useful.
As your current shaker/cacher implementation, this is not an issue for anybody then myself.
My current possible mitigations, following my modified cacher implementation also caching updates for whonix-gw-16 and whonix-wks-16, is to either:
deactivate sys-whonix’s updates-proxy-setup service so that dom0 can continue to use sys-whonix as its updatevm (and template downloader), while sys-whonix cannot report for whonix-gw-16 available updates anymore nor install packages temporarily following its whonix-gw-16 repository modifications to comply with apt-cacher-ng proxy.
Revert to your upstream cacher configuration and have whonix not having its repositories and packages cached by apt-cacher-ng (sad since whonix OS updates are common as of now across debian-11 and whonix)
Modify dom0’s /etc/qubes/repo-templates/qubes-templates.repo so that its URLs are apt-cacher-ng compliant.
The reason I got really confused (puzzled) between qubes-dom0-updates and qvm-template behavior differences can be grasped with the following:
On my current setup:
[user@dom0 ~]$ qubes-prefs updatevm
sys-whonix
[user@dom0 ~]$ qvm-service sys-whonix
qubes-update-check on
updates-proxy-setup on
Succeeds:
user@host:~$ sudo apt install htop
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Suggested packages:
lm-sensors
The following NEW packages will be installed:
htop
0 upgraded, 1 newly installed, 0 to remove and 14 not upgraded.
Need to get 127 kB of archives.
After this operation, 328 kB of additional disk space will be used.
Get:1 tor+http://HTTPS///deb.debian.org/debian bullseye/main amd64 htop amd64 3.0.5-7 [127 kB]
Fetched 127 kB in 0s (325 kB/s)
Selecting previously unselected package htop.
(Reading database ... 51973 files and directories currently installed.)
Preparing to unpack .../htop_3.0.5-7_amd64.deb ...
Unpacking htop (3.0.5-7) ...
Setting up htop (3.0.5-7) ...
Processing triggers for qubes-core-agent (4.1.37-1+deb11u1) ...
Processing triggers for desktop-file-utils (0.26-1) ...
Processing triggers for hicolor-icon-theme (0.17-2) ...
Processing triggers for man-db (2.9.4-2) ...
user@host:~$
[user@dom0 ~]$ sudo qubes-dom0-update
Using sys-whonix as UpdateVM to download updates for Dom0; this may take some time...
No updates available
So I am still puzzled onto why qubes-dom0-update would work while qvm-template wouldn’t. The mechanisms in play between qvm-template and qubes-dom0-updates is different, the latter requiring me to change my setup, which otherwise works fulfilling all my current needs, also caching whonix updates, permitting me to have ephemeral package installation into all my app qubes and also report available Templates updates to dom0 until I tried to install new templates into dom0 through any of my qubes, which relies into the same qvm-template mechanisms, where dom0 does something different than when qubes-dom0-update is called.
Basically, the question can be shortened down to:
Why qvm-template relies on dom0’s updatevm’s updates-proxy-setup (and dom0 @anyvm policy) where qubes-dom0-updates doesn’t?
qubes-dom0-update doesn’t care about appvm’s update proxy and succeeds getting dom0 available updates and reports them correctly to dom0 (talks directly to the internet through its updatevm, using dom0’s template repo definitions passed to it).
But qvm-template doesn’t work and relies on dom0’s updatevm updates-proxy-setup and dom0 updatevm policy, and would require dom0’s /etc/qubes/repo-templates/qubes-templates.repo being apt-cacher-ng compliant to be useable?
