No. You’d have to keep the USB controller where your audio device is connected in dom0.
You could apply some security policies to devices to that controller via [usbguard]. Qubes does that under certain circumstances in 4.1 as well, but it doesn’t work so well yet.
Thanks for the top @tripleh. U am very interested in using usbguard. Can you please tell us a bit more about policies with examples, or to point us to a links with such?
Thank you in advance