My first days with Qubes R4.2 rc1 install

[logoerthiner]

I decide to try R4.2 on a usb disk so that I can report bugs earlier and have most of R4.2 problems solved and ready to install R4.2 on my production machine.

I tried installing Qubes R4.2 rc1 onto my usb stick. It runs on my machine however there are various problems, big or small.

I am using btrfs and it runs good.

1. I wonder whether swap partition is required in the install, and I wonder why the swap partition is not under the same luks subtree as the main partition, causing two luks partition under the same device (cannot it be merged into one?). In R4.1 swap partition does not exist;

2. I have installed the system with non-English setup. There are too many part that are localized that surprised me, however certain part of qubes os does not seem to be aware of localization.

dom0 qubesd[3832]:     self.transport.write('{}\0{}\0'.format(k, str(v)).encode('ascii'))
dom0 qubesd[3832]:   File "/usr/lib/python3.11/site-packages/qubes/api/__init__.py", line 374, in send_event
dom0 qubesd[3832]:     self.send_event(subject, event, **kwargs)
dom0 qubesd[3832]:   File "/usr/lib/python3.11/site-packages/qubes/api/admin.py", line 70, in vm_handler
dom0 qubesd[3832]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
dom0 qubesd[3832]:     effect = func(self, event, **kwargs)
dom0 qubesd[3832]:   File "/usr/lib/python3.11/site-packages/qubes/events.py", line 164, in _fire_event
dom0 qubesd[3832]:                                   ^^^^^^^^^^^^^^^^^^^^^^^
dom0 qubesd[3832]:     sync_effects, async_effects = self._fire_event(event,
dom0 qubesd[3832]:   File "/usr/lib/python3.11/site-packages/qubes/events.py", line 227, in fire_event_async
dom0 qubesd[3832]:     await self.fire_event_async('domain-start-failed',
dom0 qubesd[3832]:   File "/usr/lib/python3.11/site-packages/qubes/vm/qubesvm.py", line 1215, in start
dom0 qubesd[3832]:     await self.dest.start()
dom0 qubesd[3832]:   File "/usr/lib/python3.11/site-packages/qubes/api/admin.py", line 862, in vm_start
dom0 qubesd[3832]:                ^^^^^^^^^^^^^^^^^^^^^^^^
dom0 qubesd[3832]:     response = await self.mgmt.execute(
dom0 qubesd[3832]:   File "/usr/lib/python3.11/site-packages/qubes/api/__init__.py", line 297, in respond
dom0 qubesd[3832]: Traceback (most recent call last):
dom0 qubesd[3832]: During handling of the above exception, another exception occurred:

(it happens when I use a non-English language and starts a sys-net that ought to pop “internal error: cannot reset PCI device xxxx:yy.z: no available FLR, PM or bus reset” in my language)

3. maybe related to btrfs. The disk usage for sys-net incorrectly accounts for the template vm size - it shows 4575MB of usage while it only has 2GB of private storage

UPDATE for 3. When I boot Qubes OS the second time, the sys-net size becomes correct. I suspect that there may be a bug somehow.

UPDATE second time for 3. When I boots sys-net, this size become incorrect again

4. sys-net cannot reset the device again and i cannot fix this with simple no strict reset, not sure why, maybe similar to the old problem in R4.1; I can only remove the offending device and restart it

UPDATE: it turns out that strict reset is the key to solve it. It is only that the GUI way of strict reset is bad.

The Qube Manager → sys-net → settings → Devices → Configure strict reset for PCI devices is buggy and not intuitive. It is buggy since the option only works when I click “OK” but not “Apply”. Intuitive way is to make strict reset (and permissive) to be a checkbox giving clear information about whether one device is currently under strict reset mode.

I was investigating qvm-device when I found this. Again this might be reported somewhen however not fixed now.

5. little aesthetic problem: Qubes Global Config app: the help snippet is helpful but some of them contains <b></b> characters which are not rendered and shows on itself. (example: updates → dom0 updates → enable all testing updates → (question mark))

(Also can be shown is that the text on OK/Cancel/Apply buttons are not visible. this is true for most of the themes. afaik it has been reported before)

6. Qubes Global Config app: this device → xen is unknown. is it good?

7. screen saver settings cannot be started

dom0

$ xscreensaver-settings 
xscreensaver-settings: 15:32:11.31: Unable to retrieve object 'manual' from class template for type 'XScreenSaverDialog' while building a 'XScreenSaverDialog'

8. suspend not working. I will investigate on this later when I have more time; basically it sleeps but not correctly resumes; it takes a long time to sleep.

UPDATE: it works perfectly today. do not know why.

9. despite I have installed R4.2 on my usb disk, it still overrides the efi boot option of the Qubes R4.1 and I need to use efibootmgr to rescue it.

Qubes 4.1: efibootmgr usage

As my stupid motherboard BIOS always looses the UEFI boot entry of Qubes once I disconnect my SSD with Qubes on it, I frequently have to run efibootmgr to fix it. BTW: Is that behavior a BIOS “bug” or is that “by design”? I knew how to do that with 4.0 based on Qubes UEFI troubleshooting guide efibootmgr -v -c -u -L Qubes -l /EFI/qubes/xen.efi -d /dev/sda -p 1 “placeholder /mapbs /noexitboot” With 4.1 however, this no longer works. I always end up skipping the GRUB boot loader part and havi…

10. sys-net default install has 400MB memory; it does not contains common utilities such as netstat and ifconfig; they are all fine and I am good with them. However on command not found error, the command_not_found_handle is invoked and dnf package list is refetched. This is meaningless on individual vm and the dnf invocation effectively ooms my sys-net. No command_not_found_handle inside such a small vm please.

11. also dark theme makes certain tray icon invisible
    

The tray icons become visible after a reboot. Anyway it is still a small annoyance.

12. multi lingual support are here. however time locales are incorrect. Incorrect time locale annoys user
    

time panel orders using rfc5322: weekday, day, month
while the correct order (as is the output of the command date in dom0 shell) should be rfc3339 style in current locale: year, month, day, weekday

13. on boot grub unicode.pf2 not found still happens. a very old problem

14. Qubes OS VM Backup compression algorithm, where can it be configured? I heard that R4.2 may allow zstd to be one of compression algorithms?

15. in /proc/cmdline of dom0, rd.net.dhcp.vendor-class=anaconda-Linux 6.1.28-1.qubes.fc37.x86_64 x86_64 I see this in dom0 linux command line. It is obviously a unexpected command line injection here. Where is it from?

16. (R4.0 R4.1 R4.2) The Qubes Application Menu (default on the top-left corner of screen) lags a while when the system freshly boots up. An aged bug that still remains to be fixed.

17. (R4.1 R4.2) /dev/xvdc is 10G while swap is 1G for each vm. what is the 9G /dev/xvdc3 for anyway?

18. qvm-run directly fails when the vm is not fully boot up. In earlier version this command should wait for vm to boot up.

19. installing vm using .iso is not direct any more. Mostly because such vm default setup becomes (pvh+default kernel) rather than (hvm+no kernel).

20. appvm based on fedora-38-minimal + … cannot boot within 60 seconds.

[2023-07-09 02:36:18] [    4.741964] blkfront: xvdd: barrier or flush: disabled; persistent grants: enabled; indirect descriptors: enabled; bounce buffer: enabled 
[2023-07-09 02:36:18] Waiting for /dev/xvda* devices... 
[2023-07-09 02:36:18] Qubes: Doing R/W setup for TemplateVM...
 [2023-07-09 02:36:19] [    6.097959]  xvdc: xvdc1 xvdc3
 [2023-07-09 02:36:21] Setting up swapspace version 1, size = 1073737728 bytes
 [2023-07-09 02:36:21] UUID=763f47ba-ce44-4dd7-96b7-b49abd812fa4 
[2023-07-09 02:36:33] Qubes: done. 
[2023-07-09 02:36:36] [   23.350408] EXT4-fs (xvda3): mounted filesystem with ordered data mode. Quota mode: none. 
[2023-07-09 02:36:36] Waiting for /dev/xvdd device... 
[2023-07-09 02:36:36] [   23.497547] /dev/xvdd: Can't open blockdev 
[2023-07-09 02:36:36] [   23.497781] EXT4-fs (xvdd): mounting ext3 file system using the ext4 subsystem
 [2023-07-09 02:36:36] [   23.498576] EXT4-fs (xvdd): mounted filesystem with ordered data mode. Quota mode: none.
 [2023-07-09 02:36:37] mount: can't read '/proc/mounts': No such file or directory 
[2023-07-09 02:36:49] [   35.667824] systemd[1]: systemd 253.5-1.fc38 running in system mode (+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP -GCRYPT +GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN -IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
 [2023-07-09 02:36:49] [   35.667964] systemd[1]: Detected virtualization xen. 
[2023-07-09 02:36:49] [   35.667994] systemd[1]: Detected architecture x86-64. 
[2023-07-09 02:36:49]  
[2023-07-09 02:36:49] Welcome to .[0;38;2;60;110;180mFedora Linux 38 (Thirty Eight).[0m! 
[2023-07-09 02:36:49]  
[2023-07-09 02:36:49] [   35.899254] systemd[1]: No hostname configured, using default hostname. 
[2023-07-09 02:36:49] [   35.899339] systemd[1]: Hostname set to <fedora>.
 [2023-07-09 02:36:49] [   36.398102] systemd[1]: bpf-lsm: BPF LSM hook not enabled in the kernel, BPF LSM not supported
 [2023-07-09 02:37:08] [   55.379181] systemd[1]: /usr/lib/systemd/system/qubes-gui-agent.service:15: Standard output type syslog is obsolete, automatically updating to journal. Please update your unit file, and consider removing the setting altogether.

what is the vm doing?

21. fedora-38-xfce ~/.bashrc is empty, which makes the default terminal prompt bare bash-5.2$ rather than with some basic info about cwd.

22. qvm-copy . seems not working

$ qvm-copy .
qfile-agent: Fatal error: File copy: “Invalid or incomplete multibyte or wide character; Last file: .” (error type: Invalid or incomplete multibyte or wide character)

TODO: testing on more apps, pulseaudio, docker, and etc; solve suspension problem and sys-net problems; make screenshots for some of the problems; find out whether btrfs features are correctly used by the system; etc.

Despite all them, R4.2 seems very familiar and similar to R4.1 on gui, with many ui improvements and it takes time to find them all.

also sys-gui is still on experiment stage now?

Written inside untrusted at qubes r4.2

[dro212]

7. screen saver settings cannot be started

I already submitted this issue on the qubes issues tracker. Issues · QubesOS/qubes-issues · GitHub

6. Qubes Global Config app: this device → xen is unknown. is it good?

I have the same issue. Let me know if you want to submit this one, otherwise I will.

Unfortunately, I cannot help with any of the other issues. These may warrant an official qubes testing issue to be filed.

[rustybird]

@logoerthiner

1. I wonder whether swap partition is required in the install, and I wonder why the swap partition is not under the same luks subtree as the main partition, causing two luks partition under the same device (cannot it be merged into one?). In R4.1 swap partition does not exist;

It’s not a difference of R4.1 vs. R4.2, but rather of the installer’s partitioning style for the LVM Thin layout vs. the Btrfs layout. FWIW the latter has the advantage of making it easier to use an ephemeral swap encryption key, although that’s not yet done automatically:

Randomized Swap

Not yet: But it’s straightforward to set this up if you’re on the Btrfs installation layout. I’ve been using a converter script since Qubes R3.0:

---

@logoerthiner

3. maybe related to btrfs. The disk usage for sys-net incorrectly accounts for the template vm size - it shows 4575MB of usage while it only has 2GB of private storage

Yep, the file-reflink driver (for Btrfs) and the lvm_thin driver have a different approach of accounting for disk usage, and it can look a little odd in Qube Manager:

Confused About Disk Usage Listed in Qube Manager - Not Adding Up

Qube Manager is a bit confusing about Disk Usage. Maybe it should be changed to just ignore the “root” volume for AppVMs: That volume only exists on disk while they are running, and because it’s a snapshot (of the TemplateVM’s “root” volume) it doesn’t really use all that much disk space. You’ll get more meaningful results by looking at the individual volumes, e.g. qvm-volume info debian-11:private qvm-volume info debian-11:root qvm-volume info A:private qvm-volume info A:root qvm-volume info …

[logoerthiner]

@dro212

I have the same issue. Let me know if you want to submit this one, otherwise I will.

I am glad if anyone else submits the issues. My only hope is that Qubes R4.2 has less issues on my machine when it is officially released. For those issues with no clues of solution and no developer attentions, I will submit the issues myself.

As there are too many problems with R4.2, I merely find here to be a suitable place to keep notes for the problems so that they are not forgotten, but actual reporting as issue is another story.

[procShield]

Changes to the UI are nice.

I have brought up updating issues with R4.2 in other posts. apt and dnf wont update over tor for me. Do you know why? Issues with UpdatesProxy.

Illumination/brightness control works again.

[logoerthiner]

Summarize

1. by design
2. seem not any more?
3. by design, btrfs size calculation logic changed
4. unsure - I will try again in future on my production machine
5. html tags reported Q4.2 Global Config GUI UI String issues · Issue #8743 · QubesOS/qubes-issues · GitHub; color fine now
6. still exist, pure aesthetic?
7. fixed
8. no any more; try again in future
9. efibootmgr should be learnt anyway to fix uefi boot options problem
10. sys-net default memory 400mb is not reasonable at all under default config anyway - when I personalized my templates, this size may be small; anyway design choice
11. still, must reboot
12. this should be manually configured - right click the clock, click “Property (P)”, and then edit clock format to be “custom”, and fill things like %a<b>%F</b>%R, this should make locale correct there
13. on my old machine boot gui merely disappeared at all, pure cli
14. zstd is not even the default installed package; seems gzip still
15. it is from /etc/default/grub, and why does it exist there from the first place?
16. always exist
17. cannot understand
18. unsure
19. untested for now
20. still this slow and this unpredictable. need ways to debug this
21. fixed?
22. still exist