USB microphone in separate sys-usb causes usb mouse to slow down?

Connecting a usb mouse and a usb microphone to separate USB controllers in separate sys-usb qubes, then using the Qubes Device manager to attach the USB microphone to a qube running openbroadcaster suddenly causes the the USB mouse to slow down.

Is there something that is shared between separate sys-usb qubes which containing separate USB controllers and direct to separate domains (the dom0 for the mouse and the openbroadcaster qube for the microphone) that could be causing this?

This still happens when recording video in openbroadcaster is disabled, so it’s not just a system load thing inside that qube.