How to keep cached packages updated with "Qubes Update"?

User Support
1

Hello,
I like to download some packages in template via apt install -y --download-only <package>, so that they can be later installed temporarily within an AppVM based on this template. This helps me to reduce the amount of needed templates. I read about /etc/qubes/post-install.d hooks, created a new, executable file 10-apt-download-only.sh within the template and pasted inside

#!/bin/sh
apt install -y --download-only vlc

, but somehow this does not get executed (judging from looking at /var/cache/apt/archives). What would be the proper way to keep these downloaded packages up-to-date with “Qubes Update” tool?

2

Hi,

You need to setup apt-cacher-ng in a qube, there are some tutorials on the forum. It works well with debian IMO, but for me it has been painful to configure for fedora…

3

(Adding to the previous answer: search the forum for “apt-cacher-ng” or simply “cacher”, and you’ll find a number of threads about how to set it up.)

1 Like
4

did you mean “apt-cacher-ng” instaed of “apr-cacher-ng”? :wink:

1 Like
5

Thanks @gonzalo-bulnes and @solene for both your answers. I am aware of apt-cacher-ng - there are two reasons against it for my scenario:

  1. Caching of packages within Whonix
  2. Additional overhead

For 1. I am not sure, if apt-cacher-ng can be integrated with Whonix. Also I don’t think clearnet cache should be mixed in any way with traffic from Whonix. And since I am only using one Whonix template and spawn multiple AppVMs from it, it might be used directly to cache needed packages with plain apt.

Regarding 2. this is the kind of overhead with separate qube and configuration I would like to avoid. Having a local template cache seems like a nice sweet spot.

I have searched for /etc/qubes/post-install.d, but could barely find anything… Is it possible at all to use apt in these scripts? That README in same directory states

All executable files with .sh suffix in this directory will be executed as
root just after template installation or update. Template VM may
not have access to the network at this time yet.

I don’t understand the last sentence: Of course template don’t have direct network access, since they shouldn’t be assigned a netvm. Isn’t that what update proxy is used for anyway?

6

apt-cacher-ng doesnt fit with what @riverboat wants to do. They were quite
specific in their requirements, and it isnt clear how apt-cacher-ng
would help. They want to cache the packages in the template, not in a
caching proxy.

There are three issues with @riverboat’s approach:

  1. apt is not recommended for scripting. Use apt-get or lower level apt calls.
  2. By default the archives in /var/cache/apt/archives are cleaned after an apt operation.
    This is set by /etc/apt/apt.conf.d/10no-cache.
    It would be possible to work round this by removing that file so that
    packages are cached in the template. This is somewhat inefficient.
    A better approach would be to switch on caching for the call, like this:
    apt-get upgrade -d -o APT::Keep-Downloaded-Packages="1" -y -q -q PACKAGE_LIST
  3. 10-apt-download-only.sh will always fail when called within
    /etc/qubes/post-install.d because the calling apt operation is still
    running, so that the new call to apt cannot get a lock for dpkg.
    I dont see a clean way round this.
    You could try forking to the new call and using some sleep in the hope
    that the original apt operation would have terminated. The issue there
    would be that if called from a Qubes update operation, the template
    would likely be shut down before the updated packages could be
    downloaded. Difficult.

In sum, I dont see any simple or reliable way of using post-install hooks.

I never presume to speak for the Qubes team. When I comment in the Forum I speak for myself.
1 Like
7

Thanks very much @unman for your elaborate answer.

Yes, exactly.

This is what I normally do to let apt - or better apt-get, as you explained in 1 - wait for existing locks and processes:

apt -o DPkg::Lock::Timeout=100 install -y --download-only <mypackage>

, but this still does not solve

I think above issue will be a show stopper, until there is a way for “Qubes Update” to wait for running processes hooked from /etc/qubes/post-install.d. Same goes for scripts in /rw/config/rc.local.

Thanks for clarification!

8

It’s quite possible that I’m completely wrong and that waiting for the
lock to appear by some mechanism is good enough ™.

I never presume to speak for the Qubes team. When I comment in the Forum I speak for myself.
9

Have tested again:

  • commented out APT::Keep-Downloaded-Packages "0"; in /etc/apt/apt.conf.d/10no-cache
  • manually triggered apt-get -o DPkg::Lock::Timeout=100 install -y --download-only vlc: package(s) in /var/cache/apt/archives survived a reboot
  • deleted that package again
  • triggered “Qubes Update” manualy to get template up-to-date
  • created /etc/qubes/post-install.d/10-apt-download-only.sh (+x) with
    #!/bin/sh
sleep 30
apt-get -o DPkg::Lock::Timeout=100 install -y --download-only vlc
  • shut template down
  • re-triggered “Qubes Update”
  • Now it became apparent, that Qubes Update directly showed a yellow checkmark and did not wait for the sleep 30 .

Seems you are right with update tool not waiting.