How to kexec into Qubes from Linux LiveCD

Just some notes:
I’m trying to kexec from Debian into Qubes OS 4.2 installer iso in QEMU with UEFI.
I’ve installed Debian 11 on QEMU disk and boot from it:

qemu-system-x86_64 -drive format=raw,if=pflash,file=/usr/share/OVMF/OVMF_CODE_4M.fd,readonly=on \
	-drive if=pflash,format=raw,unit=1,file=OVMF_VARS.fd \
	-drive file="debian.qcow2",format=qcow2 \
	-drive file="Qubes-4.2.202304081519-x86_64.iso",media=cdrom \
	-machine q35,accel=kvm:tcg \
	-rtc base=utc \
	-smp "$(nproc)" \
	-vga none \
	-device bochs-display \
	-m 2048 \
	-serial stdio \
	-device qemu-xhci,id=usb \
	-device usb-tablet \
	-device usb-kbd \
	-net none

I’ve copied initrd-latest.img and vmlinuz-latest from /images/pxeboot/ on Qubes OS 4.2 installer iso and use them with kexec.
I’ve also built Xen 4.17 with my patch using qubes-builder and got patched xen-4.17.0.gz from xen-hypervisor-4.17.0-8.fc32.x86_64.rpm package to use with kexec.
I’m kexec’ing from Debian into Qubes OS 4.2 installer iso like this:

kexec -d -l xen-4.17.0.gz -t multiboot-x86 --command-line="acpi_rsdp=0x7fb7d014 com1=115200,8n1 console=com1,vga loglvl=all guest_loglvl=all no-real-mode reboot=no vga=current edd=off sync_console=true sched_debug iommu=verbose apic_verbosity=debug console_timestamps" --module="vmlinuz-latest inst.stage2=hd:LABEL=QUBES-4-2-202304081519-X86-64 plymouth.ignore-serial-consoles" --module="initrd-latest.img"
kexec -e

If I use unpatched kexec 2.0.26 then system just reboots after executing kexec -e.
If I use patched kexec 2.0.26 with ebda hack then I’m booting into Qubes OS 4.2 installer just fine.

The ebda hack:
kexec patch: remove unneeded xen EBDA patch. · osresearch/heads@4f88f35 · GitHub

Patch
diff --git a/kexec/kexec.c b/kexec/kexec.c
index bc6ab3d..b82725b 100644
--- a/kexec/kexec.c
+++ b/kexec/kexec.c
@@ -805,6 +805,27 @@ static int my_load(const char *type, int fileind, int argc, char **argv,
 	if (sort_segments(&info) < 0) {
 		return -1;
 	}
+
+#if 1
+	// force segment 0 to have memsz == bufsz
+	// so that it won't overwrite EBDA
+	if (info.segment[0].mem == 0)
+	{
+		if (kexec_debug)
+			printf("hack ebda into segment 0!\n");
+
+		uint8_t * ebda = calloc(1, info.segment[0].memsz);
+		memcpy(ebda, info.segment[0].buf, info.segment[0].bufsz);
+		info.segment[0].bufsz = info.segment[0].memsz;
+		info.segment[0].buf = ebda;
+
+		// install some default EBDA values that are off scale,
+		// which will force Xen to use the multiboot info
+		*(uint16_t*)(ebda + 0x40e) = 0xFFFF; // segment
+		*(uint16_t*)(ebda + 0x413) = 0xFFFF; // size
+	}
+#endif
+
 	/* if purgatory is loaded update it */
 	update_purgatory(&info);
 	if (entry)

Kexec ebda hack only works for multiboot and doesn’t work for multiboot2 because segment 0 mem is not 0 for multiboot2.
The kexec with/without ebda hack works if I use QEMU with BIOS and multiboot2 (default for kexec > v2.0.22).
The kexec with ebda hack works if I use QEMU with BIOS and multiboot (kexec with -t multiboot-x86 option).
The kexec without ebda hack doesn’t work if I use QEMU with BIOS and multiboot (kexec with -t multiboot-x86 option).
The kexec with/without ebda hack doesn’t work if I use QEMU with UEFI and multiboot2 (default for kexec > v2.0.22) with Xen patch.
The kexec with ebda hack works if I use QEMU with UEFI and multiboot (kexec with -t multiboot-x86 option) with Xen patch.
The kexec without ebda hack doesn’t work if I use QEMU with UEFI and multiboot (kexec with -t multiboot-x86 option) with Xen patch.

2 Likes

Just to make sure we are at the same page @disp6252

QEMU BIOS/UEFI multiboot/multiboot2 ebda hack Xen patch kexec works
Yes BIOS multiboot2 Yes/No No Yes
Yes BIOS multiboot Yes No Yes
Yes BIOS multiboot No No No
Yes UEFI multiboot2 Yes/No Yes No
Yes UEFI multiboot Yes Yes Yes
Yes UEFI multiboot No Yes No

Is that right?

EDIT: it seems that we got rid of the patch maybe a little bit too fast. Will try to find time to investigate and report back, trying to get eyes on this.

Reminder: patch deleted from Heads for EBDA hack: kexec patch: remove unneeded xen EBDA patch. · osresearch/heads@4f88f35 · GitHub

Might be related to the problem we currently have under coreboot->heads->kexec-> xen → dom0 not picking up EFI to enable EFIFB early at EFIFB support next steps · Issue #1461 · osresearch/heads · GitHub

@disp6252 Thanks a lot for this thorough testing BTW!

The important part for us is