Yes !
I always thought when I can use my USB keyboard / mouse and I can connect to any USB storage, mic etc. that I have this USB qube already setup. This was also what I understood during the installation process which auto-detects my USB keyboard.
I underestimated the complexity of USB and mixed sys-usb and Qubes Device Manager.
To summarize a final conclusion:
When you have a Qubes OS with a USB keyboard / mouse setup and you want to use a Yubikey you simply have to run:
https://www.qubes-os.org/doc/usb-qubes/#automatic-setup
sudo qubesctl state.sls qvm.usb-keyboard
Afterwards launch the sys-usb and your are done.