Configure the system to not use the secondary GPU so it can be attached to a guest

Again, which one did you follow:
[Case: No dedicated screen, mouse or keyboard] VirtualGL and vulkan
or
[Case: Dedicated Screen, mouse and keyboard] Linux guest — Integration with QubesOS
?