I recently installed qubes exactly as what you have described, creating partitions, specifying qvm-pool, manually installing templates, etc. It all worked well, and I’m grateful for your instructions.
However, I could’t get any of VM to start. In their log, I saw that they complained about the filesystem of /xvdc, as you have described in that GitHub issue. I think that line of qvm-pool command was intentioned to avoid this ( by using lvm thin pool, as you said on GitHub), but unluckily it didn’t work for me.
Should I reinstall qubes, or should I build 4kn templates and find a way to transfer them into dom0 without any VM running? Thank you!
Btw, my self-built 4kn template also fails to start for the same reason, in qubes on a 512e ssd.
@rustybird Sorry I was not more specific: I meant for the root and private volumes creation: was that tested working?
So if I understand well, I could apply your patch and have volatile volume fixed. But for creating root volumes and private volumes, I would need to build ISO, or patch stage 1 and stage 2 install so that when templates are decompressed, those are fixed to create a working system to be able to compare performance properly with/without the fixes.
I was looking for next steps to get main devs attention in seeing actual performance losses/ differences in this thread.
Otherwise, people are trying to get away of LVM thin provisioning model at install as of now. Some wants ZFS,XFS/BRTFS since speed differences are quite important.
Fixing LUKS+LVM thin provisioning would be great. Otherwise LVM is blamed for performance losses as of now where other implementations are simply not suffering from the same implementation flaws that LVM thin provisioning is suffering from, per Qubes implementation of volatile, private and root volumes creation.
Not sure that I understand your question, but standard (i.e. not in like a standalone HVM) private volumes are already sector-size agnostic in their content, so compatibility wise it doesn’t matter whether they are presented to the VM as 512B or 4KiB block devices.
Standard root volumes have sector-size specific content, and I don’t think it’s feasible to dynamically patch that volume content (specifically, the partition table) in dom0, because it contains untrusted and potentially malicious VM controlled data.
Backward compatibility is a real headache here. It seems like the existing root and private volumes should simply be presented to the VM as 512B devices by default for now. In the case of an LVM installation layout, that might even entail forcing 512B sectors for the whole LUKS device - unless there’s a good way to set an independent sector size for the LVM pool or ideally per LVM volume.
I’d like to share some info about my playing with the 4kn drive.
I went through 51lieal’s instructions (using stock r4.1.1 iso). I got two main volumes, one is “varlibqubes” using file-reflink driver, the other is “vm” using lvm-thin driver.
When I was installing templates, I found that stock 512b templates didn’t boot on “vm”, but booted on “varlibqubes”. My self-built 4kn template booted on both two volumes.
However, since I upgraded dom0 to testing-latest, all my VMs refuse to boot, and the error messages are “libxenlight failed to create domain xxx”, just like those when I try to boot 512b templates on “vm”.
I can confirm this isn’t a kernel issue, because downgrading kernel version doesn’t help. I suspect this is somthing xen related.
Even stock “LVM” installation ( because I cannot proceed with “LVM thin” option) without those modifications to luks on a 4kn drive will lead to the same result, that VMs refuse to boot when testing-repos are enabled.
I am just a user, when this issue is solved by the tech-people I’ll just reïnstall Qubes en restore from back-up.
Or stop using Qubes on this machine because another issue is I really need to install rpmfusion repos in dom0, what is not available in EOL Fedora 32.
Edit: sorry. I think as I go and seem to not be able to do a full post one shot and always edit multiple times. Sorry if you reply from email, hopefully this is sent to you in the 10 minutes edition permission time prior of sending the email you would reply to.
It seems the reason you were thwarted by gdisk is that it only guards the first sector alignment for you. And if you always make the LUKS partition last, then it will happily make the end ‘ragged’, not sizing the partition to multiples of the alignment value.
Easy fix for this is:
In gdisk eXpert menu, choose ‘L’ and enter ‘8’ if your drive has 512b sectors. This should also work if your drive has 4096b sectors. ‘8’ is 4096 / 512.
Go back to the main menu with ‘M’ and choose ‘N’ for a new partition. The default start should be OK. For the end, specify a relative canonical value like ‘+64g’ or ‘+64800m’. Using ‘g’ or ‘m’ will give you intrinsic multiples of 4096.
After that, cryptsetup with --sector-size 4096 should not complain about alignment.