SSD maximal performance : native sector size, partition alignment

User Support Hardware Issues
7

@51lieal unfortunately, trimming (discard) and performance of disks are not testable in “vm” and require manual tweaks to be tested if performance difference is to be tested, since vm performance depends on created lvm partitions (sectors and sizes), vg created block sizes, then LUKS (most important here from my understanding) sectors and block sizes, needing to be coherent with what the firmware does internally. Not an expert whatsoever here, but intuition here seems to confirm that some disks performing better with reencryption speed have properly reported block size from OS:

[user@dom0 ~]$ cat /sys/block/sda/queue/physical_block_size 
4096

Proper testing seem to require redoing aligned partition table, aligned LUKS partition, and then reinstall (while lvm partitioning also seem to matter).
Yet again, in a faster performing drive, lvms were properly created with better alignements:

[user@dom0 ~]$ cat /sys/block/dm-142/queue/physical_block_size 
4096

It also seem to require not to reinstall from the installer directly, but to do some of the actions first from Qubes available terminals prior of going forward in the installer, and having the installer “rescan” the disks to take into account what was done outside of it to proceed in manual partitioning.

Hey! Just saw that you are the user behind the post I was going to refer: 4.1 installer LVM partitioning - hard to customize, missing space - #5 by 51lieal

Basically, applying the following differences to test

  • run cryptsetup-reencrypt from a live cd. Take total time and speed in MiB/s at final output.
  • make aligned partition table in expert mode (just notes) parted -a optimal /dev/sda mklabel gpt with block size of 4096 bytes (default alignment not tuned for special manufacturers)
  • cryptsetup -c aes-xts-plain64 -h sha512 -s 512 --use-random -y -i 10000 --sector-size 4096 luksFormat /dev/sda2
  • Follow through with the rest of the instructions to prepare custom partitions from your referred guide.
  • Install system, make sure that LVM partitions created are good by booting system
  • do cryptsetup-reencrypt again from live CD, check difference of performance and report results.
  • Adapt values above, which otherwise seem to be aligned for sector and block sizes of 512 in current observations, as reported in dom0 by: cat /sys/block/sda/queue/physical_block_size

So as of now, I can already see from observations that some SSDs are not reporting properly their block sizes, and that tooling take reported upper layer block size and apply it to the whole down chain from the installer from automatic partitioning.

It also seems that my particular problem came from having suboptimal partitioning for sector/block sizes on initial install, which were cloned from one disk to another in the past.

Those are notes… Not truth. Further experimentation only will confirm or infirm this hypothesis, while others having only changed sector sizes/block sizes/alignment on the same hardware for the same SSD have witnessed major gains in performance (no vm/Qubes usage reported though), while not reencrypting their disks either (which obviously tackle SSD hardware differences as caches, erase block sizes and firmware optimization. So testing seems the only way to validate this, on same computer with same SSDs, where I will test cloning disk of different reported block sizes (512 vs 4096) on other disk and test and report clone disks (on each other) and variations of observed performance (where cryptsetup-reencrypt being my personal meter).

2 Likes