How to kexec into Qubes from Linux LiveCD

Just some notes:
I’m trying to kexec from Debian into Qubes OS 4.2 installer iso in QEMU with UEFI.
I’ve installed Debian 11 on QEMU disk and boot from it:

qemu-system-x86_64 -drive format=raw,if=pflash,file=/usr/share/OVMF/OVMF_CODE_4M.fd,readonly=on \
	-drive if=pflash,format=raw,unit=1,file=OVMF_VARS.fd \
	-drive file="debian.qcow2",format=qcow2 \
	-drive file="Qubes-4.2.202304081519-x86_64.iso",media=cdrom \
	-machine q35,accel=kvm:tcg \
	-rtc base=utc \
	-smp "$(nproc)" \
	-vga none \
	-device bochs-display \
	-m 2048 \
	-serial stdio \
	-device qemu-xhci,id=usb \
	-device usb-tablet \
	-device usb-kbd \
	-net none

I’ve copied initrd-latest.img and vmlinuz-latest from /images/pxeboot/ on Qubes OS 4.2 installer iso and use them with kexec.
I’ve also built Xen 4.17 with my patch using qubes-builder and got patched xen-4.17.0.gz from xen-hypervisor-4.17.0-8.fc32.x86_64.rpm package to use with kexec.
I’m kexec’ing from Debian into Qubes OS 4.2 installer iso like this:

kexec -d -l xen-4.17.0.gz -t multiboot-x86 --command-line="acpi_rsdp=0x7fb7d014 com1=115200,8n1 console=com1,vga loglvl=all guest_loglvl=all no-real-mode reboot=no vga=current edd=off sync_console=true sched_debug iommu=verbose apic_verbosity=debug console_timestamps" --module="vmlinuz-latest inst.stage2=hd:LABEL=QUBES-4-2-202304081519-X86-64 plymouth.ignore-serial-consoles" --module="initrd-latest.img"
kexec -e

If I use unpatched kexec 2.0.26 then system just reboots after executing kexec -e.
If I use patched kexec 2.0.26 with ebda hack then I’m booting into Qubes OS 4.2 installer just fine.

The ebda hack:
kexec patch: remove unneeded xen EBDA patch. · osresearch/heads@4f88f35 · GitHub

Patch
diff --git a/kexec/kexec.c b/kexec/kexec.c
index bc6ab3d..b82725b 100644
--- a/kexec/kexec.c
+++ b/kexec/kexec.c
@@ -805,6 +805,27 @@ static int my_load(const char *type, int fileind, int argc, char **argv,
 	if (sort_segments(&info) < 0) {
 		return -1;
 	}
+
+#if 1
+	// force segment 0 to have memsz == bufsz
+	// so that it won't overwrite EBDA
+	if (info.segment[0].mem == 0)
+	{
+		if (kexec_debug)
+			printf("hack ebda into segment 0!\n");
+
+		uint8_t * ebda = calloc(1, info.segment[0].memsz);
+		memcpy(ebda, info.segment[0].buf, info.segment[0].bufsz);
+		info.segment[0].bufsz = info.segment[0].memsz;
+		info.segment[0].buf = ebda;
+
+		// install some default EBDA values that are off scale,
+		// which will force Xen to use the multiboot info
+		*(uint16_t*)(ebda + 0x40e) = 0xFFFF; // segment
+		*(uint16_t*)(ebda + 0x413) = 0xFFFF; // size
+	}
+#endif
+
 	/* if purgatory is loaded update it */
 	update_purgatory(&info);
 	if (entry)

Kexec ebda hack only works for multiboot and doesn’t work for multiboot2 because segment 0 mem is not 0 for multiboot2.
The kexec with/without ebda hack works if I use QEMU with BIOS and multiboot2 (default for kexec > v2.0.22).
The kexec with ebda hack works if I use QEMU with BIOS and multiboot (kexec with -t multiboot-x86 option).
The kexec without ebda hack doesn’t work if I use QEMU with BIOS and multiboot (kexec with -t multiboot-x86 option).
The kexec with/without ebda hack doesn’t work if I use QEMU with UEFI and multiboot2 (default for kexec > v2.0.22) with Xen patch.
The kexec with ebda hack works if I use QEMU with UEFI and multiboot (kexec with -t multiboot-x86 option) with Xen patch.
The kexec without ebda hack doesn’t work if I use QEMU with UEFI and multiboot (kexec with -t multiboot-x86 option) with Xen patch.

2 Likes

Just to make sure we are at the same page @disp6252

QEMU BIOS/UEFI multiboot/multiboot2 ebda hack Xen patch kexec works
Yes BIOS multiboot2 Yes/No No Yes
Yes BIOS multiboot Yes No Yes
Yes BIOS multiboot No No No
Yes UEFI multiboot2 Yes/No Yes No
Yes UEFI multiboot Yes Yes Yes
Yes UEFI multiboot No Yes No

Is that right?

EDIT: it seems that we got rid of the patch maybe a little bit too fast. Will try to find time to investigate and report back, trying to get eyes on this.

Reminder: patch deleted from Heads for EBDA hack: kexec patch: remove unneeded xen EBDA patch. · linuxboot/heads@4f88f35 · GitHub

Might be related to the problem we currently have under coreboot->heads->kexec-> xen → dom0 not picking up EFI to enable EFIFB early at EFIFB support next steps · Issue #1461 · linuxboot/heads · GitHub

@disp6252 Thanks a lot for this thorough testing BTW!

The important part for us is

Sorry for the late response.

That’s right.

Also I forgot to mention the important note - with Xen patch the RSDP is passed to Xen properly and Xen works ok, but Xen doesn’t pass it further to dom0 kernel so dom0 has some problems.
It’s possible to patch dom0 kernel so it’ll be possible to pass the RSDP to it the same way but I didn’t try to do this.
Here is an example on how to do this:
https://lore.kernel.org/all/20161202195416.58953-1-andriy.shevchenko@linux.intel.com/T/#u

1 Like