SSD maximal performance : native sector size, partition alignment

I’d like to share some info about my playing with the 4kn drive.
I went through 51lieal’s instructions (using stock r4.1.1 iso). I got two main volumes, one is “varlibqubes” using file-reflink driver, the other is “vm” using lvm-thin driver.

When I was installing templates, I found that stock 512b templates didn’t boot on “vm”, but booted on “varlibqubes”. My self-built 4kn template booted on both two volumes.

However, since I upgraded dom0 to testing-latest, all my VMs refuse to boot, and the error messages are “libxenlight failed to create domain xxx”, just like those when I try to boot 512b templates on “vm”.

I can confirm this isn’t a kernel issue, because downgrading kernel version doesn’t help. I suspect this is somthing xen related.

Even stock “LVM” installation ( because I cannot proceed with “LVM thin” option) without those modifications to luks on a 4kn drive will lead to the same result, that VMs refuse to boot when testing-repos are enabled.

Was that tested with xmm-xen from trsting repos? (With without directio patch?)

I am confused since that patch was removed in latest xmm-xen, which is why I’m asking.

Testing thread Xen + xmm-xen fixes to test (suspend/resume fixes, directio loopback devices, sys-usb fails to start, slowness vs bare metal) - #5 by Insurgo

Sorry just saw that this issue was referred under VMs don't boot on 4Kn drive · Issue #7828 · QubesOS/qubes-issues · GitHub

Important discussions happening there.

Important advancements here VM partition layout expects 512B sector size, VMs fail to start from 4096B drive · Issue #4974 · QubesOS/qubes-issues · GitHub

Props to @rustybird ! Thanks!!!

What is your SSD? 102mb/s for cryptsetup-reencrypt seems a bit on the lower end of speed?

Eh, it’s an SATA Samsung 850 Pro on an old ThinkPad T420. :person_shrugging:

I’ve not using xfs+lvm-thin anymore so i don’t have idea, currently using btrfs, and there’s no problem with it (i’m using testing repo).

Yes, Yes, Marmarek just reverted “Use direct-io for loop device” in xen 4.14.5-10, which should be in testing repo now.

$ cat /sys/block/nvme0n1/queue/physical_block_size
$ cat /sys/block/nvme1n1/queue/physical_block_size

QubesOS 4.1 on two nvme SSD’s. Disk model WDC WDS500G2B0C-00PXH0 (x2)
Notebook model: NH5x_NH7xHP serial: N/A UEFI: INSYDE v: 1.07.01
CPU: Single Core 11th Gen Intel Core i7-11800H speed: 2304 MHz

Extreme slow boot, something like 6 minutes:
@dom0 ~]$ systemd-analyze blame
49.894s qubes-vm@sys-firewall.service >
43.064s qubes-vm@sys-usb.service >
32.900s dracut-initqueue.service >
31.298s systemd-cryptsetup@luks\x2d05b49888\x2df626\x2d47c1\x2d9dde\x2d26bfd00f>
28.157s qubes-vm@sys-net.service >
27.710s systemd-cryptsetup@luks\x2daeb29ae5\x2d9b10\x2d4de2\x2d8665\x2d579831c5>
15.741s qubesd.service >
15.663s upower.service >
15.569s systemd-logind.service >
8.355s systemd-udev-settle.service >
7.327s lvm2-monitor.service >
4.793s plymouth-quit-wait.service >
3.263s lvm2-pvscan@253:0.service >
2.218s systemd-journal-flush.service >
1.680s dracut-cmdline.service >
840ms qubes-qmemman.service

@dom0 ~]$ systemd-analyze critical-chain
The time when unit became active or started is printed after the “@” character.
The time the unit took to start is printed after the “+” character. @1min 18.259s
└─ @1min 18.258s
└─qubes-vm@sys-firewall.service @28.363s +49.894s
└─qubes-meminfo-writer-dom0.service @28.354s
└─qubes-core.service @27.996s +330ms
└─qubesd.service @12.227s +15.741s
└─lvm2-pvscan@253:0.service @8.957s +3.263s
└─system-lvm2\x2dpvscan.slice @8.891s

I am just a user, when this issue is solved by the tech-people I’ll just reïnstall Qubes en restore from back-up.
Or stop using Qubes on this machine because another issue is I really need to install rpmfusion repos in dom0, what is not available in EOL Fedora 32.

Oh! That was online reencryption, never did that!

I lost track on github issues with last comment being that open-qa fails as expected at

What are the next steps @rustybird ?

In link with your comment VMs don't boot on 4Kn drive · Issue #7828 · QubesOS/qubes-issues · GitHub

Edit: sorry. I think as I go and seem to not be able to do a full post one shot and always edit multiple times. Sorry if you reply from email, hopefully this is sent to you in the 10 minutes edition permission time prior of sending the email you would reply to.

It’s pretty neat.

I’m not sure. For the time being, people with 4Kn drives should probably simply install Qubes OS with the Btrfs installation layout.

@rustybird unfortunately, not a desirable path for everything thin-lvm related support, including wyng support :confused:

1 Like