First Installation attempt with Qubes release 4.1.2 resulted in output of 4 or 5 lines and stucks over minutes with blinking Cursor.
Second Installation attempt with release 4.2.0-rc4 produced this output.
Web search of the relevant parts “acpi_ev_address_space_dispatch” “__die_body.cold” leads me to
opened 08:37PM - 24 Oct 23 UTC
closed 03:04PM - 30 Oct 23 UTC
**Describe the bug**
I tried to boot a cloud-hypervisor VM on a machine that'… s previously ran KVM/Xen guests fine and got the following kernel panic:
```
Oct 24 10:52:07 host1 microvm@vm1[15894]: [ 0.413970] BUG: unable to handle page fault for address: ffffc9000002d000
Oct 24 10:52:07 host1 microvm@vm1[15894]: [ 0.414047] #PF: supervisor write access in kernel mode
Oct 24 10:52:07 host1 microvm@vm1[15894]: [ 0.414047] #PF: error_code(0x000b) - reserved bit violation
Oct 24 10:52:07 host1 microvm@vm1[15894]: [ 0.414047] PGD 3400067 P4D 3400067 PUD 35a7067 PMD 35a8067 PTE 800000fffffee163
Oct 24 10:52:07 host1 microvm@vm1[15894]: [ 0.414047] Oops: 000b [#1] PREEMPT SMP NOPTI
Oct 24 10:52:07 host1 microvm@vm1[15894]: [ 0.414047] CPU: 1 PID: 1 Comm: swapper/0 Not tainted 6.1.59 #1-NixOS
Oct 24 10:52:07 host1 microvm@vm1[15894]: [ 0.414047] Hardware name: Cloud Hypervisor cloud-hypervisor, BIOS 0
Oct 24 10:52:07 host1 microvm@vm1[15894]: [ 0.414047] RIP: 0010:acpi_ex_system_memory_space_handler+0x29a/0x310
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] Code: ff ff 0f b6 00 49 89 06 31 c0 e9 2b fe ff ff 49 8b 16 88 10 31 c0 e9 1f fe ff ff 8b 00 49 89 06 31 c0 e9 13 fe ff ff 49 8b 16 <89> 10 31 c0 e9 07 fe ff ff 41 89 e8 48 c1 ed 20 45 89 f9 48 c7 c2
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] RSP: 0018:ffffc90000013970 EFLAGS: 00010246
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] RAX: ffffc9000002d000 RBX: 0000000000000020 RCX: 0000000000000000
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] RDX: 0000000000000000 RSI: ffff88800355a900 RDI: ffffffff827b84a0
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] RBP: 000000fffffee000 R08: ffffffff827b84c0 R09: ffffc9000002dfff
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] R10: ffff888003d1c840 R11: ffffc9000002d000 R12: ffff888003d1c660
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] R13: 0000000000000001 R14: ffffc90000013aa0 R15: 000000000000000c
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] FS: 0000000000000000(0000) GS:ffff88800eb00000(0000) knlGS:0000000000000000
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] CR2: ffffc9000002d000 CR3: 0000000002610000 CR4: 00000000000406e0
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] Call Trace:
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] <TASK>
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] ? __die_body.cold+0x1a/0x1f
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] ? page_fault_oops+0xd2/0x2b0
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] ? exc_page_fault+0xae/0x150
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] ? asm_exc_page_fault+0x22/0x30
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] ? acpi_ex_system_memory_space_handler+0x29a/0x310
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] acpi_ev_address_space_dispatch+0x134/0x340
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] ? acpi_ex_prep_field_value+0x2e0/0x2e0
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] acpi_ex_access_region+0xcd/0x270
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] acpi_ex_field_datum_io+0x7d/0x2c0
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] acpi_ex_write_with_update_rule+0x9d/0x100
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] acpi_ex_insert_into_field+0x29a/0x380
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] acpi_ex_write_data_to_field+0x70/0x140
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] acpi_ex_store_object_to_node+0x172/0x230
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] ? acpi_ex_resolve_to_value+0x1f5/0x2c0
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] acpi_ex_opcode_1A_1T_1R+0x254/0x610
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] acpi_ds_exec_end_op+0x27e/0x510
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] acpi_ps_parse_loop+0xf7/0x6a0
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] acpi_ps_parse_aml+0x7c/0x3d0
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] acpi_ps_execute_method+0x13b/0x270
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] acpi_ns_evaluate+0x1ee/0x2d0
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] acpi_ut_evaluate_object+0x64/0x1c0
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] acpi_ut_execute_STA+0x39/0x80
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] acpi_ns_get_device_callback+0xfe/0x1d0
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] ? _raw_spin_unlock_irqrestore+0x23/0x40
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] ? preempt_count_add+0x6a/0xa0
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] acpi_ns_walk_namespace+0x1d0/0x260
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] ? _raw_spin_unlock_irqrestore+0x23/0x40
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] ? acpi_get_devices+0xc0/0xc0
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] acpi_get_devices+0xa4/0xc0
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] ? set_no_mwait+0x27/0x27
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] acpi_init+0x349/0x4cf
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] ? acpi_sleep_proc_init+0x24/0x24
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] do_one_initcall+0x59/0x220
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] kernel_init_freeable+0x1eb/0x228
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] ? rest_init+0xd0/0xd0
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] kernel_init+0x16/0x130
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] ret_from_fork+0x22/0x30
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] </TASK>
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] Modules linked in:
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] CR2: ffffc9000002d000
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] ---[ end trace 0000000000000000 ]---
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] RIP: 0010:acpi_ex_system_memory_space_handler+0x29a/0x310
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] Code: ff ff 0f b6 00 49 89 06 31 c0 e9 2b fe ff ff 49 8b 16 88 10 31 c0 e9 1f fe ff ff 8b 00 49 89 06 31 c0 e9 13 fe ff ff 49 8b 16 <89> 10 31 c0 e9 07 fe ff ff 41 89 e8 48 c1 ed 20 45 89 f9 48 c7 c2
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] RSP: 0018:ffffc90000013970 EFLAGS: 00010246
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] RAX: ffffc9000002d000 RBX: 0000000000000020 RCX: 0000000000000000
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] RDX: 0000000000000000 RSI: ffff88800355a900 RDI: ffffffff827b84a0
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] RBP: 000000fffffee000 R08: ffffffff827b84c0 R09: ffffc9000002dfff
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] R10: ffff888003d1c840 R11: ffffc9000002d000 R12: ffff888003d1c660
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] R13: 0000000000000001 R14: ffffc90000013aa0 R15: 000000000000000c
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] FS: 0000000000000000(0000) GS:ffff88800eb00000(0000) knlGS:0000000000000000
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] CR2: ffffc9000002d000 CR3: 0000000002610000 CR4: 00000000000406e0
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.414047] note: swapper/0[1] exited with irqs disabled
Oct 24 10:52:08 host1 microvm@vm1[15894]: [ 0.837254] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000009
```
**Version**
Output of `cloud-hypervisor --version`: 32.1
Did you build from source, if so build command line (e.g. features):
**VM configuration**
What command line did you run (or JSON config data):
```
/nix/store/vl6m6cjhyzzlsz7jc60faxkjpyr8697s-cloud-hypervisor-32.1/bin/cloud-hypervisor --cpus boot=2 --watchdog --console null --serial tty --kernel /nix/store/dc6qmalzs44bmha4yb9xzww8c8xfj9sc-linux-6.1.59-dev/vmlinux --initramfs /nix/store/b707dqlfknfga3j28m3c9f0sa6d9n8c9-initrd-linux-6.1.59/initrd --cmdline console=ttyS0 reboot=t panic=-1 loglevel=4 init=/nix/store/9hak5np6327palic37baahq6b3v4g3hw-nixos-system-vm2-23.05pre-git/init regInfo=/nix/store/axiclqwvn8i436k099pwyhvgnrj5x5fq-closure-info/registration --seccomp true --memory mergeable=on,shared=on,size=1024M --disk path=vm1-var.img --fs tag=ro-store,socket=vm2-virtiofs-ro-store.sock --api-socket vm2.sock --net tap=vm-vm1,mac=...
```
Guest OS version details: NixOS 23.05
Host OS version details: NixOS 23.05
Linux kernel output: 6.1.59
There was stated “Try --cpus max_phys_bits=39 or 38. We place our MMIO regions at the end of the guest physical address space - we have seen some CPUs from AMD wrongly report the address space size.”
My interpretation of ‘guest physical address-space’ ist the total cpu physical address-space of 48 bits minus the bits used for MMIO. Which is returned by the CPU microcode to the kernel function acpi_ev_address_space_dispatch().
Assuming that my amd cpu microcode contains a similar bug even it is a modern Zen 4 architecture. Is there a chance to address the bug in Xen or Linux kernel via command line arguments in Grub boot manager similar to KVM Hypervisor command line args ’ --cpu max_phys_bits= ’ ?
szz9pza
November 12, 2023, 10:48am
2
Hi,
According to your picture, you already have the latest BIOS for your model.
(Just writing this for possible future readers:
You could try to update your BIOS to the latest version.)
In the link you posted, the acpi
problem has been solved by adding the kernel option acpi=off
.
The --cpus max_phys_bits=
is for the next problem that this user encounter about virtio device.
This post might be useful: https://forum.qubes-os.org/t/thinkpad-t16-amd-ryzen-acpi-issue-no-keyboard-no-touchpad/21685
It report that acpi must be disabled (acpi=off
) to install Qubes.
It also have posted an other forum topic.
this one: https://forum.qubes-os.org/t/my-adventures-with-qubes-4-1-1-on-a-lenovo-t14-gen-3/14370
Maybe that will help you.
2 Likes
Hi,
that hint is was helpful. Thanks!
My first attempt to install with the acpi=off with the ‘kernel-latest’ boot option resulted in an endless loop
‘watchdog: BUG: soft lockup -CPU#0 stuck for 25s! [init:1]’ counting +25s per loop.
The second attempt choosing the first GRUB boot option gets me to the graphical install screen.
Only the linux kernel parameter acpi=off was needed.
I’m impressed that you hit the mark. Before posting here I tried several xen and kernel params suggested in thes Qubes OS User Support forum without success.
In order to proceed I have to obtain an usb keyboard/mouse first as the internal devices are not functional…
1 Like
@methusalix I’m glad you were able to progress!
A little pro tip: I marked @szz9pza ’s post as the solution, so future folks (and our future selves!) can see that the topic includes a solution, and get a highlight of that solution in the first post.
If you want yo do that in the future, it’s something that you can do yourself by using the little “checkbox” icon at the bottom of the post that you want to mark as the solution. Welcome to the forum!