Have you tried something like this?
Note that a more elegant solution would be to create a second sys-usb with android tools and use it only for Android debugging.