My goal here is to get boot printk’s to understand a hardware compat issue, so first I’m looking for info on how to setup a boottime serial console with qubes.
From what I saw:
console= kernel parameters are ignored, and we see through dmesg that printk get directed to tty0 and hvc0
some information exist on qubes website (Safe Remote Dom0 Terminals | Qubes OS), but they are only applicable to a running system, and won’t help getting early kernel logs
Then, as is not uncommon those days on a laptop, I do not have an RS232 interface. The kernel does support console=ttyUSB to cope with this, but how we can make it work in the qubes model, where we cannot even attach a USB/serial device do dom0, is not yet clear to me either.
Any hint to get those early printk’s to the outside world ?
You can, of course, attach a USB device to dom0, by removing the
“rd.qubes.hide_all_usb” from the boot parameters.
If you have a USB3 debug port, DbC is designed to log printk messages -
you’ll need a debugging kernel, of course, but it will give you
something before the console comes onstream.
plug a USB-A/USB-A cable to a port on this controller (you may need to go trial-and-error, or just enable on all controllers and see) with the other end to another computer, check kernel logs – if all goes well you should see:
… and on the other end of the cable you should see a new USB device appear, if if you have the proper driver available it should bind to /dev/ttyUSB0: the xHCI host controller was really replaced with a device.
activate on Qubes
QubesOS kernels have CONFIG_EARLY_PRINTK_USB_XDBC=y defined already, ready for the job.
(WiP)
We need to:
allow dom0 do get the xHCI controller (remove rd.qubes.hide_all_usb)
prevent sys-usb from stealing it (add qubes.skip_autostart)
direct kernel messages to DbC (add earlyprintk=xdbc, or =xdbc1 to select the second controller – the core also looks for the undocumented keep string)
current status
for now all I could get from earlyprintk is as follows (here for xdbc1, on AMD Renoir 4800H) :
I hoped to be activate the device as console afterwards, like we can do with /sys/class/tty/ttyS0/console, but even as /sys/class/tty/ttyDBC0/ appears, in gets no console control.
late console
All I’m able to get for now is a late dom0 console through an external USB/serial adapter and console=ttyUSB0,115200.
While getting logs on serial line for dom0 can be useful, when investigating early boot failures it can also be valuable to extract the logs from the hypervisor itself.
Xen has support for EHCI DBGP (USB2) but as of 4.14/4.16 no support yet for XHCI DBC. So recent laptops without an EHCI controller are out here.
For those with EHCI support, I could find no HOWTO (only the Xen commandline doc) so let’s summarize here: you will need to:
activate the DBGP driver on the controller you mean to use
specificy dbgp as console
The commandline options for Xen (not for the dom0 linux kernel) would be, once you have identified the BDF address of your PCI EHCI, something like:
dbgp=ehci@pci7:0.3 console=dbgp,vga
Disclaimer: could not test it yet, only have a XHCI laptop available, for now
In addition even if you have a USB2 laptop with debug support on one of the ports (which is supposedly supported by Xen), there are no currently on the market products that support connecting a USB2 debug port to a monitoring host. I’ve looked. The last (high-end) one on the market is mentioned here in 2015, but does not appear to be sold any more: Xen project Mailing List
To discover whether a USB2 controller supports such a debug port, see output of lspci -v|less and search for a line with ‘ebug’ in one of the USB controller blocks.
Other ideas:
If the device has an externally facing PCIE port (e.g. expresscard, or a desktop with PCIE slots), a PCIE serial port card (as long as it’s real and not a PCIE->USB->serial combo device) may work, depending upon Xen (and possibly BIOS/UEFI). This is supposedly supported by Xen.
Another approach which I haven’t seen attempted, would be to set up a thunderbolt machine with the bios thunderbolt settings set to the least secure settings, connected to a PCIE enclosure with a PCIE serial port adapter. Again, this may depend on BIOS/UEFI support.
IIRC, Marek did not have luck with an M.2 NVME → PCIE serial port adapter, though unclear if that was a BIOS limitation.
Also, if you have a server (or a business laptop) with Intel AMT or V-PRO, you can always use the Serial Over LAN functionality for Xen or Dom0 debugging.
For obvious reasons, you want to have AMT disabled if it’s a feature of your hardware build. I keep AMT disabled in the BIOS of the Lenovo W520s and W530s I have…but I did a test for you, dear reader.
If I set it up in BIOS, a new ttyS port appears in dom0. I then shut down and disabled it.
Xen docs say you can use the port info available in dom0 to construct the Xen command line (to start with this port as the debug console). You will need another machine on the same ethernet LAN segment and will need to dig out the information you need from this guide: Xen Serial Console - Xen
