I know what you’re trying to say with these questions, trying to “infer” some sense of security based on the statistics, but since you have the completed products - why not analyze directly?? Read through the packages, remove as necessary, build a salt script to automate, or if your interested build from source.
Most likely these are generic chosen based on distro / gnome / xfce standards.
Look inside the template_debian and template_rpm folders, they contain all the scripts and package lists related to the build process (most of it is documented inside the scripts).
Some packages are here because of dependencies, others for qubes-related tasks or to match the distribution itself.
To make an even lighter template, you could build it yourself with qubes-builderv2 and remove some packages from the .lists files if they don’t break the template at the end.
I’m not sure about dom0, I’ll have to check again.
Well, not really. I have looked there before opening the thread. All I can see is that an incremental approach is used after some bootstrapping procedures. That may be part of the answer to question 1. It is still not obvious what the starting point is though, so even that question needs overall clarification.
I found another place related to template building:
It seems to mount an image and install things in it. From what I understand after looking at it for a few minutes, it seems to build templates from scratch.
Here’s the first 200 lines from the building process of a debian bookworm template, just in case you want to analyze them a bit:
Executing 'sudo mkdir -p /builder /builder/build /builder/plugins /builder/distfiles&&sudo chown -R user:user /builder&&make -C /builder/plugins/template prepare build-rootimg'.
output: make: Entering directory '/builder/plugins/template'
output: Building template: debian-12 (202312121118)
output: mkdir -p /builder/build
output: echo 202312121118 > /builder/build/build_timestamp_debian-12
output: # Check that required env are defined
output: sudo env -i DIST_CODENAME="bookworm" DIST_NAME="debian" DIST_VER="12" PLUGINS_DIR="/builder/plugins" ARTIFACTS_DIR="/builder/build" CACHE_DIR="/builder/cache/cache_bookworm" TEMPLATE_CONTENT_DIR="/builder/sources/builder-debian/template_debian" TEMPLATE_NAME="debian-12" TEMPLATE_VERSION="4.2.0" PACKAGES_DIR="/builder/repository" TEMPLATE_SCRIPTS_DIR="/builder/plugins/template/scripts" KEYS_DIR="/builder/plugins/chroot_deb/keys" TEMPLATE_FLAVOR="" TEMPLATE_OPTIONS=" firmware" TEMPLATE_FLAVOR_DIR="+:/builder/sources/builder-debian/template_debian/ +firmware:/builder/sources/builder-debian/template_debian/firmware" APPMENUS_DIR="" CONFIG_DIR="" TEMPLATE_CONF="/builder/build/template.conf" VERBOSE="1" DEBUG="1" PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" DISCARD_PREPARED_IMAGE="1" TEMPLATE_ROOT_WITH_PARTITIONS="1" TEMPLATE_ROOT_SIZE="20G" USE_QUBES_REPO_VERSION="" USE_QUBES_REPO_TESTING="" BUILDER_TURBO_MODE="1" REPO_PROXY="" FEDORA_MIRROR="" CENTOS_MIRROR="" EPEL_MIRROR="" QUBES_MIRROR="" GENTOO_MIRROR="" ARCHLINUX_MIRROR="" FLAVORS_DIR="" RELEASE="4.2" DIST="bookworm" DISTRIBUTION="debian" /builder/plugins/template/scripts/prepare-image /builder/build/prepared_images/debian-12.img /builder/mnt
output: + IMG=/builder/build/prepared_images/debian-12.img
output: ++ readlink -m /builder/mnt
output: + INSTALL_DIR=/builder/mnt
output: ++ dirname /builder/plugins/template/scripts/prepare-image
output: + BUILDER_SCRIPTS_DIR=/builder/plugins/template/scripts
output: + LC_ALL=POSIX
output: + RETCODE=0
output: + . /builder/plugins/template/scripts/builder-setup
output: ++ REQUIRED_ENV=(DIST_CODENAME DIST_NAME DIST_VER PLUGINS_DIR ARTIFACTS_DIR CACHE_DIR TEMPLATE_CONTENT_DIR TEMPLATE_NAME TEMPLATE_SCRIPTS_DIR KEYS_DIR)
output: ++ for var in "${REQUIRED_ENV[@]}"
output: ++ '[' -z bookworm ']'
output: ++ for var in "${REQUIRED_ENV[@]}"
output: ++ '[' -z debian ']'
output: ++ for var in "${REQUIRED_ENV[@]}"
output: ++ '[' -z 12 ']'
output: ++ for var in "${REQUIRED_ENV[@]}"
output: ++ '[' -z /builder/plugins ']'
output: ++ for var in "${REQUIRED_ENV[@]}"
output: ++ '[' -z /builder/build ']'
output: ++ for var in "${REQUIRED_ENV[@]}"
output: ++ '[' -z /builder/cache/cache_bookworm ']'
output: ++ for var in "${REQUIRED_ENV[@]}"
output: ++ '[' -z /builder/sources/builder-debian/template_debian ']'
output: ++ for var in "${REQUIRED_ENV[@]}"
output: ++ '[' -z debian-12 ']'
output: ++ for var in "${REQUIRED_ENV[@]}"
output: ++ '[' -z /builder/plugins/template/scripts ']'
output: ++ for var in "${REQUIRED_ENV[@]}"
output: ++ '[' -z /builder/plugins/chroot_deb/keys ']'
output: +++ id -ur
output: ++ [[ 0 != 0 ]]
output: + . /builder/plugins/template/scripts/umount-kill
output: ++ '[' 1 == 1 ']'
output: ++ set -x
output: +++ basename /builder/plugins/template/scripts/prepare-image
output: ++ '[' prepare-image == umount-kill ']'
output: + '[' 2 -ne 2 ']'
output: + '[' -z 20G ']'
output: + mkdir -p /builder/mnt /builder/cache/cache_bookworm /builder/repository /builder/build
output: + export INSTALL_DIR LC_ALL IMG
output: ++ df -T /dev
output: ++ tail -1
output: ++ cut -f 1 -d ' '
output: + '[' tmpfs = tmpfs ']'
output: + mount -t devtmpfs none /dev
output: + DIST_TO_STR=bookworm+
output: + '[' 9 -gt 0 ']'
output: + DIST_TO_STR='bookworm+ (options: firmware)'
output: + echo 'INFO: Preparing installation of bookworm+ (options: firmware) template...'
output: + /builder/sources/builder-debian/template_debian/00_prepare.sh
output: INFO: Preparing installation of bookworm+ (options: firmware) template...
output: ++++ basename /builder/sources/builder-debian/template_debian/00_prepare.sh
output: +++ '[' 00_prepare.sh == umount-kill ']'
output: ++ output 'INFO: /builder/sources/builder-debian/template_debian/distribution.sh imported by: /builder/sources/builder-debian/template_debian/00_prepare.sh'
output: ++ '[' 01 -ge 1 ']'
output: ++ [[ -z '' ]]
output: ++ [[ ehB != \e\h\x\B ]]
output: + umount_all /builder/mnt
output: + directory=/builder/mnt
output: + '[' /builder/mnt == /builder/mnt ']'
output: ++ mountPoints /builder/mnt
output: ++ local mount_point
output: +++ mountPoint /builder/mnt
output: +++ local mount_point=/builder/mnt
output: +++ [[ /builder/mnt == /* ]]
output: +++ echo /builder/mnt
output: +++ sed 's#//*#/#g'
output: ++ mount_point=/builder/mnt
output: +++ sudo grep /builder/mnt /proc/mounts
output: +++ cut -f2 '-d '
output: +++ sort -r
output: +++ grep '^/builder/mnt'
output: +++ uniq
output: ++ echo ''
output: + '[' -n '' ']'
output: + umount_kill /builder/mnt
output: + '[' 01 -le 2 ']'
output: + test -o xtrace
output: + true 'umount_kill: Disabling xtrace, because variable VERBOSE (1) is lower than or equal 2...'
output: + set +x
output: INFO: Attempting to kill any processes still running in '/builder/mnt' before un-mounting
output: + true 'umount_kill: Restoring xtrace...'
output: + buildStep /builder/sources/builder-debian/template_debian/00_prepare.sh pre
output: + local filename=/builder/sources/builder-debian/template_debian/00_prepare.sh
output: + local suffix=pre
output: + unset build_step_files
output: + info 'Locating buildStep files: 00_prepare.sh suffix: pre'
output: + output 'INFO: Locating buildStep files: 00_prepare.sh suffix: pre'
output: + '[' 01 -ge 1 ']'
output: + [[ -z '' ]]
output: + [[ ehB != \e\h\x\B ]]
output: + getFileLocations build_step_files /builder/sources/builder-debian/template_debian/00_prepare.sh pre
output: + local return_global_var=build_step_files
output: + local filename=/builder/sources/builder-debian/template_debian/00_prepare.sh
output: + local suffix=pre
output: + local function=templateFile
output: + unset GLOBAL_CACHE
output: + declare -gA GLOBAL_CACHE
output: + callTemplateFunction /builder/sources/builder-debian/template_debian/00_prepare.sh pre templateFile
output: + local calling_script=/builder/sources/builder-debian/template_debian/00_prepare.sh
output: + local calling_arg=pre
output: + local functionExec=templateFile
output: + local template_flavor=
output: + local template_options
output: + templateFile /builder/sources/builder-debian/template_debian/00_prepare.sh pre ''
output: + local file=/builder/sources/builder-debian/template_debian/00_prepare.sh
output: + local suffix=pre
output: + local template_flavor=
output: + local template_dirs
output: ++ templateDirs ''
output: ++ local template_flavor=
output: ++ local template_flavor_prefix
output: ++ local template_flavor_dir
output: ++ local match=0
output: ++ read -r -a template_flavor_dir
output: ++ for element in "${template_flavor_dir[@]}"
output: +++ templateName ''
output: +++ local template_flavor=
output: +++ local template_name
output: +++ local template_options
output: +++ local template_label
output: +++ local template_options_concatenated
output: +++ retval=1
output: +++ read -r -a template_options
output: +++ '[' -n '' ']'
output: +++ '[' -z ' firmware' ']'
output: ++++ printf +%s firmware
output: +++ template_options_concatenated=+firmware
output: ++++ templateFlavorPrefix ''
output: ++++ local template_flavor=
output: ++++ local template_flavor_prefix
output: ++++ read -r -a template_flavor_prefix
output: ++++ '[' '' == + ']'
output: ++++ echo bookworm
output: +++ template_name=bookworm+firmware
output: +++ read -r -a template_label
output: ++++ templateNameFixLength bookworm+firmware
output: ++++ local template_name=bookworm+firmware
output: ++++ local temp_name
output: ++++ read -r -a temp_name
output: ++++ local index=1
output: ++++ '[' 17 -ge 32 ']'
output: ++++ echo bookworm+firmware
output: +++ echo bookworm+firmware
output: +++ return 1
output: ++ '[' + == bookworm+firmware ']'
output: ++ '[' + == + ']'
output: ++ '[' + == + ']'
output: ++ eval echo -e /builder/sources/builder-debian/template_debian/
output: +++ echo -e /builder/sources/builder-debian/template_debian/
output: ++ match=1
output: ++ for element in "${template_flavor_dir[@]}"
output: +++ templateName ''
output: +++ local template_flavor=
output: +++ local template_name
output: +++ local template_options
output: +++ local template_label
output: +++ local template_options_concatenated
output: +++ retval=1
output: +++ read -r -a template_options
output: +++ '[' -n '' ']'
output: +++ '[' -z ' firmware' ']'
output: ++++ printf +%s firmware
output: +++ template_options_concatenated=+firmware
output: ++++ templateFlavorPrefix ''
output: ++++ local template_flavor=
output: ++++ local template_flavor_prefix
output: ++++ read -r -a template_flavor_prefix
output: ++++ '[' '' == + ']'
output: ++++ echo bookworm
output: +++ template_name=bookworm+firmware
output: +++ read -r -a template_label
output: ++++ templateNameFixLength bookworm+firmware
output: ++++ local template_name=bookworm+firmware
output: ++++ local temp_name
output: ++++ read -r -a temp_name
output: ++++ local index=1
output: ++++ '[' 17 -ge 32 ']'
output: ++++ echo bookworm+firmware
output: +++ echo bookworm+firmware
output: +++ return 1
output: ++ '[' +firmware == bookworm+firmware ']'
output: ++ '[' + == + ']'
output: ++ '[' +firmware == + ']'
output: ++ '[' + == '*' ']'
output: ++ '[' 1 -eq 1 ']'
output: ++ return
output: + template_dirs=/builder/sources/builder-debian/template_debian/
output: + splitPath /builder/sources/builder-debian/template_debian/00_prepare.sh path_parts
output: + local return_global_var=path_parts
output: + local filename=00_prepare.sh
output: + local dir=/builder/sources/builder-debian/template_debian/
output: + local base=00_prepare
Hi, just one human to another here- I recognize your account across threads, you ask interesting questions. I feel though your response posts often read as brusque and impolite, ungrateful. I think this is unintentional! I wanted to point your attention to it though, as I’m often interested in reading the answers to your questions yet I feel your follow-ups have a chilling effect on getting to good, deeper answers. For this stranger’s sake, please try to be kinder to people who take the time to respond to you, even when their answers are unsatisfying to you.
(I would have written this as a private message but I couldn’t figure out how to do it. Maybe my account is too new.)
please try to be kinder to people who take the time to respond to you
Based on how some recent threads developed, you can understand why I choose my replies to be selectively terse. Time flies for all of us and there is nothing unkind if all participants consider that. On the contrary. I hope you would agree.
Thanks all for keeping the conversation civil! Feedback is not always easy to give and receive, and it’s great to see that happening smoothly.
I’ve followed up privately on how to send private messages if there is interest in continuing this side-conversation, and I’d suggest bringing the public thread back on topic.
After digging a little deeper into Debian’s template in particular, I think I have clarified this part:
some bootstrapping procedures
The function bootstrap in qubes-builder-debian/template_debian/01_install_core.sh calls debootstrap, which is a Debian specific utility that installs required and important packages, as explained here. As seen in the same function, the command also adds some additional packages listed in the --include parameter. I guess we can conclude that at least for Debian things start from a base system and build incrementally.
For Fedora (qubes-builder-rpm) it is still difficult for me to understand what the base is. I notice --exclude options in the packages_*.list files (even in the _minimal versions).
I suspect that the forum software changed my answer, which was this -
The latter.
bootstrap a basic system, and install packages on it, as you have seen.
make a usable system, providing (most of) expected programs
Follow the culture of the distro, while attempting to make templates
cohere.
The core team - although reasoned arguments for inclusion/changes are
welcome.
a. Yes
b. Some people decrement, others increment from micro base. You should
also look at use of microkernels.
c. Headless qubes providing single service - e.g. firewall, Tor, VPN.
Micro qubes providing minimal GUI services.
Thank you! It is so good to see someone providing meaningful answers.
Micro qubes providing minimal GUI services.
Could you show any examples of that where I can see how it is done? I have already looked at a project called Liteqube but it is really not the most readable code (I have spent several days trying to understand it).
How is dom0 created?
What are you asking
6a. Does it start from the same base used for the Fedora templates, or from something else (what)?
6b. How is it constructed as a whole (in general) and what is different compared to constructing a template, or to installing a Linux system on bare metal?
6c. Are the qubes-specific packages also distro-specific, i.e. are they written only for Fedora, or is it possible to create a dom0 based on a different distro without adapting the source code to it first?
The GitHub issue linked in the first post of the following topic talks about that. My take away is that few things if any are distro-specific, and _most\ tools include info to be packages as both RPM and Debian packages. (For some value of few and most!)
(And I’d encourage you to separate that question from this topic, my two cents.)
