Hey guys, thanks for the replies @FireLaser @necker
im back after locking myself out and needing to borrow a ps2 keyboard, hard to track those down these days 
I have successfully found a way to change the template of sys-usb though
qvm-shutdown --wait sys-usb; qvm-prefs -s sys-usb template new-template-dvm; qvm-start sys-usb
Ive also found all my usb devices are working on a fedora-34-minimal template including my yubikey.
my issue was my fedora-34-dvm was broken and not routing through my new sys-vpn. I created a new dvm which worked but couldn’t deleted the old dvm because it was in use by sys-usb.
Ive been wanting to shift sys-usb to a minimal template so this all worked out for the best 