Really confused. Are you able to install any template as of today since integration of qvm-template?!
EDIT: seems like qubes-dom0-update talks correctly through update proxy of sys-whonix/sys-firewall, but that qubes-repo-templates package and logic of appvm reuses that appvms proxy-forwarder if one is defined.
Follow example in issuecomment-1300743530
In Dom0:
[user@dom0 ~]$ qubes-prefs updatevm
sys-whonix
In whonix-gw-16
user@host:~$ sudo apt install qubes-repo-templates
...
Setting up qubes-repo-templates (4.1.2-1+deb11u1) ...
user@host:~$
In Dom0:
[user@dom0 ~]$ qvm-shutdown sys-whonix whonix-gw-16 --force --wait
[user@dom0 ~]$ sudo qubes-dom0-update qubes-template-fedora-36
Redirecting to 'qvm-template install fedora-36'
Downloading 'qubes-template-fedora-36-0:4.0.6-202205270243'...
qubes-template-fedora-36-0:4.0.6-202205270243 ...
Note: No fedora-36 installed prior test.
Is qubes-repo-templates required now in updatevm?
Seems so per reffered issue.
Sorry for the noise, my setup is special: I have whonix templates updates cached by a modified shaker/cacher apt-cacher-ng setup.
My issue is that sys-whonix repositories are modifed to be conform to apt-cacher-ng format (http://HTTPS/// urls in repository related files) where qubes-repo-templates package basically deploys new repository file definition and GPG keys, and that file was not modified to be compliant with apt-cacher-ng expected format.
Note that shaker/cacher currently excludes whonix templates support and that my bug affects me and me alone, having sys-whonix check for updates related to whonix-gw-16 (qubes-update-check service) and have sys-whonix use apt-cacher-ng proxy per dom0 policy and activation through updates-proxy-setup service activated).
My setup is not a recommended deployment, while whonix support under shaker/cacher is discussed over @unman shaker/cacher relative issue.
I do not see @unman’s shaker/cacher being impacted by this issue if sys-whonix is left alone by cacher and left for dom0 to use it as dom0 updatevm (option at install).
@unman: I see a possible issue though if sys-firewall is used as dom0 updatevm and if sys-firewall is used to report dom0 updates, while sys-firewall has updates-proxy-setup enabled with dom0 policy permitting @anyvm to talk through apt-cacher-ng. As I remember the conclusion of our talks, you were against doing that, expecting users to update all templates prior of usage, so no template updates notifications to users. Otherwise, users enabling update checks through cacher will need both inotifywatch daemon and cacher_configure script to rewrite repository definitions under /etc/qubes/repo-templates/qubes-templates.repo, which is a new requirements of qvm-template but currently not deployed under Qubes template
[user@dom0 ~]$ qvm-service sys-firewall
qubes-update-check on
updates-proxy-setup on
From having updates-proxy-setup service enabled for dom0 updatevm, that would mean that updatevm would have dom0 template downloaded through apt-cacher-ng as well and then passed to dom0’s updatevm… I am reconsidering having updates notifications when using cacher and not sure how to go forward.
So in the scope of general user deployment qvm-template fails with unhelpful error if qubes-repo-templates not installed in UpdateVM · Issue #7605 · QubesOS/qubes-issues · GitHub current mitigation requires the user to manually install qubes-repo-templates in dom0’s defined updatevm’s Template.
For general user deployment this is not true.
That is, for Debian and Fedora based qubes not currently need to be
installed.
For minimal templates (advanced users only) - issue?documentation required?
For Whonix? What is the real cause of problem?
identification of the actual issue would be useful,
I never presume to speak for the Qubes team.If can help please tell how. sys-whonix updatevm here no cacher. Test for Insurgo because he treasure. unman also extraordinary precious wizard so happy if can help. Now have qubes-repo-templates install in whonix-gw-16. Should revert? What tests useful?
I have not extensively tested this. I have just witnessed that qubes-repo-templates package is not installed under debian/fedora/whonix, as referred in currently opened issue, with mea culpa telling my setup was the cause of my issue (cacher with repositories all modified to be compatible with apt-cacher-ng. Therefore, newly deployed repositories by qubes-repo-templates package in templates were not apt-cacher-ng compatible and the requests from those templates simply silently failed, until I deactivated updates-proxy-setup service, with qubes-repo-templates package installed to infirm my prior hypothesis. I configured my qubes to check and report to dom0 for available updates, and to do so, I need those templates to be apt-cacher-ng compatible, and use apt-cacher-ng update proxy through enabling updates-proxy-setup on those qubes. By doing so, not modifying /etc/qubes/repo-templates/qubes-templates.repo, deployed by qubes-repo-templates package results in dom0 qvm-template failing. Other configuration will oviously have different result, while qubes-repo-templates package still needs to be deployed and Qubes documentation updated accordingly and where shaker/cacher currently is not affected by the bad behavior, simply because cacher use case relies on users manually updating templates and not relaying on qubes reporting available updates (qubes-update-check service on qubes) nor having qubes rely on apt-cacher-ng to report such updates nor use apt-cacher-ng to fetch repository packages (updates-proxy-setup is nly enabled on Templates through shaker/cacher. App qubes, such as sys-firewall if defined as qubes dom0 updatevm, will be used through normal https connection and won’t fail qvm-template calls from dom0 if cacher is not used. @unman : makes sense?
Conclusion is confirming that qvm-template is lacking proper error reporting in case qubes-repo-templates is not deployed on dom0’s updatevm Template, current Qubes OS documentation not referring to qvm-template at all, while requirements of a additioanl qubes-repo-templates package deployment under templates is nowhere to be found, nor automatically installed.
@Confused : can you validate that
[user@dom0 ~]$ qvm-prefs sys-firewall template
What is template?
If template Fedora/Debian without qubes-repo-templates package installed in that template:
[user@dom0 ~]$ qubes-prefs updatevm sys-firewall
[user@dom0 ~]$ qvm-template install fedora-37
And report back?
[user@dom0 ~]$ qvm-prefs sys-firewall template
sys-firewall-dvm
[user@dom0 ~]$ qvm-prefs sys-firewall-dvm template
ks-16-min-fw
Note: ks-16-min-fw is debian-11-minimal distro-morph to kicksecure-qubes-cli without qubes-repo-templates
[user@dom0 ~]$ qubes-prefs updatevm
sys-whonix
[user@dom0 ~]$ qubes-prefs updatevm sys-firewall
[user@dom0 ~]$ qvm-template install fedora-37
[Qrexec] /etc/qubes-rpc/qubes.TemplateSearch: line 3: /usr/lib/qubes/qvm-template-repo-query: No such file or directory
ERROR: qrexec call 'qubes.TemplateSearch' failed.
[user@dom0 ~]$ qubes-prefs updatevm sys-whonix
[user@dom0 ~]$ qvm-template install fedora-37
usage: qvm-template [--verbose] [--quiet] [--help] [--repo-files REPO_FILES] [--keyring KEYRING] [--updatevm UPDATEVM] [--enablerepo REPOID] [--disablerepo REPOID]
[--repoid REPOS] [--releasever RELEASEVER] [--refresh] [--cachedir CACHEDIR] [--keep-cache] [--yes]
{install,reinstall,downgrade,upgrade,download,list,info,search,remove,purge,clean,repolist,migrate-from-rpmdb} ...
qvm-template: error: Template 'fedora-37' not found.
[user@dom0 ~]$ qubes-prefs updatevm
sys-whonix
[user@dom0 ~]$ qvm-template list
Installed Templates
debian-11-minimal 0:4.0.6-202108191622 qubes-templates-itl
fedora-32 0:4.0.6-202010192324 qubes-templates-itl
fedora-32-minimal 0:4.0.6-202010191916 qubes-templates-itl
fedora-36 0:4.0.6-202205270243 qubes-templates-itl
fedora-36-minimal 0:4.0.6-202205270243 qubes-templates-itl
whonix-gw-16 0:4.0.6-202206270941 qubes-templates-community-testing
whonix-ws-16 0:4.0.6-202206270840 qubes-templates-community-testing
Available Templates
debian-10-minimal 0:4.0.6-202010131933 qubes-templates-itl
debian-10 0:4.0.6-202009131420 qubes-templates-itl
debian-11-minimal 0:4.0.6-202108191622 qubes-templates-itl
debian-11 0:4.0.6-202201171556 qubes-templates-itl
fedora-32-minimal 0:4.0.6-202010191916 qubes-templates-itl
fedora-32-xfce 0:4.0.6-202010191916 qubes-templates-itl
fedora-32 0:4.0.6-202010192324 qubes-templates-itl
fedora-33-minimal 0:4.0.6-202102131637 qubes-templates-itl
fedora-33-xfce 0:4.0.6-202102140100 qubes-templates-itl
fedora-33 0:4.0.6-202102131637 qubes-templates-itl
fedora-34-minimal 0:4.0.6-202110020922 qubes-templates-itl
fedora-34-xfce 0:4.0.6-202110020922 qubes-templates-itl
fedora-34 0:4.0.6-202201160401 qubes-templates-itl
fedora-35-minimal 0:4.0.6-202205081759 qubes-templates-itl
fedora-35-xfce 0:4.0.6-202205081759 qubes-templates-itl
fedora-35 0:4.0.6-202205081759 qubes-templates-itl
fedora-36-minimal 0:4.0.6-202205270243 qubes-templates-itl
fedora-36-xfce 0:4.0.6-202205270243 qubes-templates-itl
fedora-36 0:4.0.6-202205270243 qubes-templates-itl
@Insurgo should I enable qubes-templates-itl-testing or did output help?
Well, in the above, debian-11-minimal is not expected to have qubes-repo-templates package deployed.
Here you confirm that sys-whonix, based on whonix-16-gw, was dom0 updatevm
You switched it to sys-firewall, based on
Interestingly, here, qvm-template’s Qrexec output hints about qvm-template-repo-query not having proper
And fails. I guess this is linked with qvm-template fails with unhelpful error if qubes-repo-templates not installed in UpdateVM · Issue #7605 · QubesOS/qubes-issues · GitHub directly, since you are using a minimal template and qubes-repo-templates package is not deployed there.
Then you did:
Which didn’t give any output, which is what I confirmed being bogus since whonix-gw-16, Fedora-36 and Debian-11 standard templates should have qubes-repo-templates package installed by default. Not sure what @unman meant there:
Since from your sole current testing, you confirmed that whonix-gw-16 is not deploying qubes-repo-templates package, where I confirmed that it was not deployed under fedora-36.
A quick check on debian-11 template:
user@debian-11:~$ sudo apt install qubes-repo-templates
[...]
After this operation, 19.5 kB of additional disk space will be used.
Get:1 http://HTTPS///deb.qubes-os.org/r4.1/vm bullseye/main amd64 qubes-repo-templates amd64 4.1.2-1+deb11u1 [5,028 B]
Fetched 5,028 B in 0s (24.4 kB/s)
Selecting previously unselected package qubes-repo-templates.
(Reading database ... 167991 files and directories currently installed.)
Preparing to unpack .../qubes-repo-templates_4.1.2-1+deb11u1_amd64.deb ...
Unpacking qubes-repo-templates (4.1.2-1+deb11u1) ...
Setting up qubes-repo-templates (4.1.2-1+deb11u1) ...
Confirms that qubes-repo-templates packages should be installed in all templates, otherwise its a bug.
My conclusion is that qvm-template being used under dom0 is currently missing an expected package dependency all standard deployed templates (whonix-gw-16+, debian-11+ and fedora-36+), that is, qubes-repo-templates deployed as default.
Or the repository file should come with what provides /usr/lib/qubes/qvm-template-repo-query script: qubes-core-agent-dom0-updates, or qubes-core-agent-dom0-updates having a dependency on qubes-repo-templates packages for all distributions.
Edit: Crosslinked this post from qvm-template fails with unhelpful error if qubes-repo-templates not installed in UpdateVM · Issue #7605 · QubesOS/qubes-issues · GitHub and documented issue there for devels and upstream documentation to be corrected. The way I understand it, qvm-template doesn’t work as of today for anybody but the ones having qubes-repo-templates deployed manually. This sure is a bug.
What worries me is that you are going to mislead users, based on your
own individual (broken?) set-up, and lack of systematic testing.
Test with a vanilla install.
qvm-clone debian-11-minimal template-updater
qvm-create updater -t template-updater -l purple
Update a template? OK
qubes-prefs updatevm updater
Restart updater. Confirm that qubes-repo-templates is not installed , and
there are no definitions of dom0 template repositories under /etc/qubes
In dom0, run qvm-template list. OK
qvm-template-gui, install a template. OK
Update a template? OK
In dom0, run qvm-template list. OK
qvm-template-gui, install a template. OK
The only package required to act as updatevm, beside networking, is
qubes-core-agent-dom0-updates.
Both debian-11 and fedora-36 templates can be used as base for updatevm
qube, and work without adding qubes-repo-templates package.
I am not saying there is no issue. I am saying you have not identified
the issue, and have generalised to a faulty conclusion.
You should change the title and edit the first entry to make the position
clear.
Hi,
I’m suffering of similar troubles after installing cacher and I can confirm that in my cacher qube isn’t installed qubes-core-agent-dom0-updates.
Do You think installing it may solve my troubles?
Can I do any other check You think may be useful for clarify the problem?
Thanks a lot,
cheers,
M.
@unman : i will retest again next week.
I clarified my setup, where other users missing qubes-repo-templates package (the repository file for templates repositories is provided by quebes-repo-templates under dom0’s updatevm’s template)
I agree this lacks systemic testing.
Was not yet able to install new template following your extra-space additional thin LVM, since additional paths under dom0 are used by qvm-template as opposed to qubes-dom0-update (dom0’s ~/.cache is used to store piped rpm to dom0, as well as /var/tmp paths for extraction). As of now, my wyng metadata cache under dom0 is consuming a lot of space which prevents me to test qvm-template download systematically from clean setup).
Hi Mila
Simply installing cacher should not (and in most cases does not) cause
problem.
What exactly is your set-up?
What is your updatevm - qubes-prefs or Global Settings GUI.
What is template for updatevm?
Have you made any other changes?
What are your “similar troubles”?
@Insurgo - I’ll go back and see if I understand your setup,and try to
create it.
“my wyng metadata cache under dom0 is consuming a lot of space” - good
grief. Can’t it be stored off-qube?
Hi unman,
sorry, I’ve mispelled it.
When talking about troubles, I’m talking about templates not cached by cacher qube (as oel 8 and similiar).
I’ve tried putting proxy in yum.conf, but it still cannot reach external internet.
$ qubes-prefs updatevm
sys-whonix
but I’ve yet tried to put it to somewhat else, without succeding.
Thanks for Your help!
Cheers,
M.
I’d like to add (custom) content of my 30-user.policy:
qubes.UpdatesProxy * oel-8-template @defafult allow target=sys-whonix
(before last line (with target=cacher))
Thanks again.
M.
I can wipe it off and ask for it to be synced back prior of next backup through arch-init from.
Will test again next week, but as of now I’m really puzzled by you saying quebes-repo-templates not being needed. What that package provides is repo definition for dom0 templates + gpg signatures, nothing more or less.
@mila this is off topic. Other threads are related to cacher, this thread mixed it in only because my setup had that mixed in.
Please post your issue in cacher/apt-cacher-ng threads where it is fit.
Here we try to isolate qvm-template dom0’ updatevm’ template package installation requirements and discuss about them only.
Why puzzled?
If you use a Debian updatevm, do you install the fedora 32 dom0 repository
definitions?
If you add a new repository in dom0, do you add that definition in the updatevm?
Puzzled exactly because of that.
Because qubes-dom0-update and qvm-template are not calling the same codepaths, and because qubes-repo-templates package is explicitly dom0 templates repo definitions to be added to dom0’s updatevm’s Template and exist for debian, fedora, and because rpc calls depend on the /etc/qubes/templates-repo/repo to be defined under dom0 updatevm template, and are fedora-32 related because of dom0. Have you checked what qubes-repo-template provides on debian/fedora templates?
I will check again like said but now there is something I do not understand for sure with qvm-template requirements and will read code linked to dom0 rpc calls linked to qvm-template. I checked quickly with and without qubes-repo-templates, thought i understood but now I definitely doubt with you saying that package is not required.