libvirt.libvirtError: internal error: libxenlight failed to attach pci device 0000:04:00.0

Your GPU can consist of several PCI devices and you need to passthrough all of them: