dGPU restricted to dom0/sys-gui-dom0: (security downgrade)?

Question forms two parts:
R4.0.4 - dGPU attached only to dom0, never before used & never attached to any other VM
R4.1 - dGPU attached only to sys-gui-dom0, never before used & never attached to any other VM

Disclaimer

I know there are 101 GPU passthrough threads, but I understand the implications - which is why I’m asking this specific question, because I haven’t seen it specifically answered. (Most advice I’m aware of is in general response to newbie threads asking about passthrough i.e.: GPU bad or GPU good?). If it’s been asked elsewhere I apologise for the dupe.

Is there any security downside to this setup (R4.0.4 & R4.1 respectively)?

FWIW @Demi asked on the mailing list about GPU to windows gaming HVM - so AFAIK from what she has implied R/E usb etc passthrough is already to dom0/sys-usb, there shouldn’t be any actual security downside, (or am I missing something here)?