it was /srv/formulas/base/virtual-machines-formula/qvm/ (sorry i was trying to remember the path)
This is happen when you try to run qubesctl --all state.highstate command and you have ‘enabled’ top file of sys-gui, to resolve the issue you should remove first.
Thanks! I disabled the sys-gui and now the above mentioned error is gone and there is no error message regarding the state files (10 succeeded, failed 0) but I get an exit code 20 nonetheless. The mgmt-sys-gui-gpu.log is like this:
2021-06-30 17:00:50,263 output: sys-gui-gpu:
2021-06-30 17:00:50,265 output: ----------
2021-06-30 17:00:50,266 output: _error:
2021-06-30 17:00:50,266 output: Failed to return clean data
2021-06-30 17:00:50,266 output: retcode:
2021-06-30 17:00:50,266 output: 126
2021-06-30 17:00:50,266 output: stderr:
2021-06-30 17:00:50,267 output: Request refused
2021-06-30 17:00:50,267 output: stdout:
2021-06-30 17:00:50,267 exit code: 20
When sys-gui-gpu is starting there is this error:
No such file or directory 'usr/bin/qubes-input-trigger --all --dom0'
The most basic way is creating a new .conf file under /etc/X11/xorg.conf.d and specifying:
Section "Device"
Identifer "intel graphics"
Driver "intel"
EndSection
See man xorg.conf in case there are additional options you might want to specify.
I might have gotten closer to the promised land.
I had an older install of Qubes 4.1 on another SSD, pre Xen 4.14.1 to be precise. I recalled that at some point I could start sys-gui-gpu and so I tried again yesterday. The template was still fedora-32-xfce, everything like in the first thread opened by @panati last year:
Thanks to the information trickeling in bits and pieces from here and there I think I know a little more (not much) on how this is supposed to work.
I disabled sys-gui and enabled sys-gui-gpu.
I tried attaching my 2nd GPU via qubesctl state.sls qvm.sys-gui-gpu-attach-gpu like explained in the documentation draft. This selected 00:02.0 PCI (...) Processor root port which I think isn’t the right choice. I got an error message regarding a “non-ending pci device (…)”. I don’t know what to make of it but I read that the whole sys-gui-gpu is meant to be working out of the box for Intel rather than AMD graphics. So I looked at the list of pci-devices in my sys-gui-gpu app.
My 2nd GPU is listed as a display controller for my Radeon HD 8570.
01:00.0 Display Controller: Advanced Micro Devices, Inc. [AMD/ATI] Sun PRO [Radeon HD 8570A/8570M]
I selected this one and my laptop froze immediately after applying and restarting sys-gui-gpu. I read that this is expected and a reboot might be needed.
I restarted and my autostart-apps are booting up including sys-gui-gpu.
Now I am asking myself how I can check if things are working as they should?
First of all, might this be the right choice for my 2nd GPU. (It looked right to me but what do I know?)
Do I have to specifically assign selected or all apps to the sys-gui-gpu as guiVM like with sys-gui?
Something like qvm-prefs work guivm sys-gui-gpu ?
Thanks in advance!
Just a little addition to my monologue:
So it looked like it was working the past few days until an hour ago.
When I booted up everything looked normal but then the task bar with all system icons disappeared and sys-gui-gpu doesn’t start anymore. The same libxenlight error I had with every other test system before prevents sys-gui-gpu from starting. Bummer!
After clicking a few of the system tools and most of the stuff working perfectly there was an error window regarding freedesktop.org.
After rebooting everything (=taskbar & symbols) is working again, including sys-gui-gpu. Looking at the service entry says it’s loaded and working.
I noticed that my root volume is almost full, so the problems might be connected to that.
There are quite several threads mentionning those problems with sys-gui-gpu, but I can’t really find an “official statement” about what we should do to get it running. Here is my attempt to put everything in one place at the same time I’m doing the steps. Hopefully a dev in-the-know will tell if this is what we’re expected to do, and fill in the blanks 
@fepitre or @marmarek maybe ?
I went directly to sys-gpu-gui without going the sys-gui path first, as that hybrid approach is not I’m after.
The steps go like:
dummy-psu-dom0qubesctl top.enable qvm.sys-gui-gpu
qubesctl top.enable qvm.sys-gui-gpu pillar=True
qubesctl --all state.highstate
/srv/formulas/base/virtual-machines-formula/qvm/sys-gui-gpu.sls to replace dummy-psu-dom0 with dummy-psu-sender
qubesctl calls (though I guess the 3rd one could be sufficient):
WARNING:root:warning: !compat-4.0 directive in file /etc/qubes/policy.d/35-compat.policy line 16 is transitional and will be deprecated
INFO:policy:qrexec: qubes.VMRootShell+: disp-mgmt-sys-gui-gpu -> sys-gui-gpu: allowed to sys-gui-gpu
Traceback (most recent call last):
File "/usr/bin/qrexec-policy-exec", line 5, in <module>
sys.exit(main())
File "/usr/lib/python3.8/site-packages/qrexec/tools/qrexec_policy_exec.py", line 221, in main
return asyncio.run(handle_request(
File "/usr/lib64/python3.8/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/lib64/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "/usr/lib/python3.8/site-packages/qrexec/tools/qrexec_policy_exec.py", line 274, in handle_request
await resolution.execute(caller_ident)
File "/usr/lib/python3.8/site-packages/qrexec/tools/qrexec_policy_exec.py", line 169, in execute
await super().execute(caller_ident)
File "/usr/lib/python3.8/site-packages/qrexec/tools/qrexec_policy_exec.py", line 133, in execute
await super().execute(caller_ident)
File "/usr/lib/python3.8/site-packages/qrexec/policy/parser.py", line 538, in execute
self.ensure_target_running()
File "/usr/lib/python3.8/site-packages/qrexec/policy/parser.py", line 579, in ensure_target_running
utils.qubesd_call(self.target, 'admin.vm.Start')
File "/usr/lib/python3.8/site-packages/qrexec/utils.py", line 95, in qubesd_call
raise AssertionError(
AssertionError: invalid qubesd response: b''
2021-07-30 13:57:01.566 qrexec-daemon[47761]: qrexec-daemon.c:1212:main: qrexec-agent has disconnected
domain dead
2021-07-30 13:57:01.567 qrexec-daemon[47761]: qrexec-daemon.c:1104:handle_agent_restart: cannot connect to qrexec agent: No such process
2021-07-30 13:57:01.567 qrexec-daemon[47761]: qrexec-daemon.c:1214:main: Failed to reconnect to qrexec-agent, terminating
That kinda looks like a protocol mismatch between qubesd and qrexec-agent ?
qubes-input-trigger in popups (which Discourse Search strangely locates in no other post, but which Duckduckgo luckily confirms occurs in other posts)default-mgmt-dvm from fedora-32 to fedora-33-xfce
qubesctl --all state.highstate
'/usr/bin/qubes-input-trigger --all --dom0'.input-trigger --all --dom0 issue/usr/lib/python3.8/site-packages/qubes/ext/gui.py as mentionned here
systemctl restart qubesd.service
/usr/bin/qubes-input-trigger not being found (though it is included in the template), and the qrexec_policy_exec exception persistsAt this point when I plug a USB mouse, Qubes tries to start sys-gui-gpu, which fails because of qubes-input-trigger, and my mouse is not usable. The following step probably has to be added somewhere in the recipe:
qubesctl state.sls qvm.sys-usbqubesctl top.disable qvm.sys-gui-gpuqvm-prefs work guivm sys-gui-gpu ?This is likely just a workaround for the failure to configure sys-gui-gpu: while this does fix the loss of USB mouse, it what it does is undoing one thing that qubesctl --all state.highstate does:
----------
ID: sys-usb-input-proxy
Function: file.prepend
Name: /etc/qubes-rpc/policy/qubes.InputMouse
Result: True
Comment: Prepended 1 lines
Started: 00:21:49.493232
Duration: 1.833 ms
Changes:
----------
diff:
---
+++
@@ -1,2 +1,3 @@
+sys-usb dom0 allow,user=root
sys-usb dom0 allow,user=root,target=sys-gui-gpu
$anyvm $anyvm deny
Aside from this, after updating to current 4.1 snapshot I still have those same symptoms: @marmarek @fepitre qrexec-agent seems to crash in sys-gui-gpu, what kind of information could I collect ? Wouldn’t this script benefit from more logging ?
… so let’s fill in the blanks.
Rerunning on nowadays 4.1beta, the last problem blocking sys-gui-gpu installation was `No such file or directory: '/usr/bin/qubes-input-trigger'` in dom0 trying to configure `sys-gui-gpu` · Issue #6893 · QubesOS/qubes-issues · GitHub, which can be worked around by installing qubes-input-proxy-sender by hand in dom0.
Then comes the time to attach the GPU and reboot. Be ready to have you QubesOS loop-rebooting: you may need to add qubes.skip_autostart on kernel command-line to be able to boot again, until you find the proper way to attach your GPU.
I tried to test sys-gui on latest kernel 4th Sep 2021 weekly build and I got the following errors at the last command. Where should I go from here. Please guide. I am clueless.
[Anirban@dom0 ~]$ sudo qubesctl top.enable qvm.sys-gui
local:
----------
qvm.sys-gui.top:
----------
status:
enabled
[Anirban@dom0 ~]$ sudo qubesctl top.enable qvm.sys-gui pillar=True
local:
----------
qvm.sys-gui.top:
----------
status:
enabled
[Anirban@dom0 ~]$ sudo qubesctl --all state.highstate
[ERROR ] Command ‘systemd-run’ failed with return code: 1
[ERROR ] stdout: Running scope as unit: run-rc28f9fd9937b4a92af56de82f7460ead.scope
Using sys-firewall as UpdateVM to download updates for Dom0; this may take some time…
Running ‘/usr/lib/qubes/qubes-download-dom0-updates.sh --doit --nogui ‘–exclude=qubes-template-whonix-gw-15,qubes-template-debian-10,qubes-template-fedora-34,qubes-template-whonix-ws-15’ ‘-y’ ‘–best’ ‘–allowerasing’ ‘–clean’ ‘–action=install’ ‘qubes-template-fedora-34-xfce’’ on sys-firewall
sys-firewall: command failed with code: 1
[ERROR ] retcode: 1
[ERROR ] Error occurred installing package(s). Additional info follows:
errors:
- Running scope as unit: run-rc28f9fd9937b4a92af56de82f7460ead.scope
Using sys-firewall as UpdateVM to download updates for Dom0; this may take some time…
Running ‘/usr/lib/qubes/qubes-download-dom0-updates.sh --doit --nogui ‘–exclude=qubes-template-whonix-gw-15,qubes-template-debian-10,qubes-template-fedora-34,qubes-template-whonix-ws-15’ ‘-y’ ‘–best’ ‘–allowerasing’ ‘–clean’ ‘–action=install’ ‘qubes-template-fedora-34-xfce’’ on sys-firewall
sys-firewall: command failed with code: 1
[ERROR ] ====== [‘present’] ======
====== stderr ======
/usr/bin/qvm-create sys-gui --class=AppVM --template=fedora-34-xfce --label=black
app: Error creating VM: Got empty response from qubesd. See journalctl in dom0 for details.
====== [‘prefs’] ======
Virtual Machine does not exist!
====== [‘service’] ======
[SKIP] Skipping due to previous failure!
ID: topd-always-passes
Function: test.succeed_without_changes
Name: foo
Result: True
Comment: Success!
Started: 20:01:57.951700
Duration: 0.984 ms
Changes:
ID: qubes-template-fedora-34-xfce
Function: pkg.installed
Result: False
Comment: Error occurred installing package(s). Additional info follows:
errors:
- Running scope as unit: run-rc28f9fd9937b4a92af56de82f7460ead.scope
Using sys-firewall as UpdateVM to download updates for Dom0; this may take some time...
Running '/usr/lib/qubes/qubes-download-dom0-updates.sh --doit --nogui '--exclude=qubes-template-whonix-gw-15,qubes-template-debian-10,qubes-template-fedora-34,qubes-template-whonix-ws-15' '-y' '--best' '--allowerasing' '--clean' '--action=install' 'qubes-template-fedora-34-xfce'' on sys-firewall
sys-firewall: command failed with code: 1
Started: 20:02:02.712592
Duration: 292687.428 ms
Changes:
ID: dummy-psu-sender
Function: pkg.installed
Result: True
Comment: The following packages were installed/updated: dummy-psu-sender
Started: 20:06:55.400359
Duration: 26406.807 ms
Changes:
----------
dummy-psu-sender:
----------
new:
1.0.0-2.fc32
old:
ID: dummy-backlight-dom0
Function: pkg.installed
Result: True
Comment: 1 targeted package was installed/updated.
Started: 20:07:21.849464
Duration: 24626.707 ms
Changes:
----------
dummy-backlight-dom0:
----------
new:
1.0.0-2
old:
ID: /usr/share/xsessions/sys-gui.desktop
Function: file.managed
Result: True
Comment: File /usr/share/xsessions/sys-gui.desktop updated
Started: 20:07:46.507336
Duration: 82.714 ms
Changes:
----------
diff:
New file
ID: sys-gui
Function: qvm.vm
Result: False
Comment: ====== ['present'] ======
====== stderr ======
/usr/bin/qvm-create sys-gui --class=AppVM --template=fedora-34-xfce --label=black
app: Error creating VM: Got empty response from qubesd. See journalctl in dom0 for details.
====== ['prefs'] ======
Virtual Machine does not exist!
====== ['service'] ======
[SKIP] Skipping due to previous failure!
====== ['features'] ======
[SKIP] Skipping due to previous failure!
Started: 20:07:46.603628
Duration: 818.917 ms
Changes:
ID: sys-gui-rpc
Function: file.managed
Name: /etc/qubes/policy.d/50-gui-sys-gui.policy
Result: True
Comment: File /etc/qubes/policy.d/50-gui-sys-gui.policy updated
Started: 20:07:47.422933
Duration: 21.854 ms
Changes:
----------
diff:
New file
ID: sys-gui-admin-local-rwx
Function: file.append
Name: /etc/qubes/policy.d/include/admin-local-rwx
Result: True
Comment: Appended 2 lines
Started: 20:07:47.445158
Duration: 21.312 ms
Changes:
----------
diff:
---
+++
@@ -8,3 +8,5 @@
## Add your entries here, make sure to append "target=dom0" to all allow/ask actions
+sys-gui @tag:guivm-sys-gui allow target=dom0
+sys-gui sys-gui allow target=dom0
ID: sys-gui-admin-global-rwx
Function: file.append
Name: /etc/qubes/policy.d/include/admin-global-rwx
Result: True
Comment: Appended 3 lines
Started: 20:07:47.466618
Duration: 3.329 ms
Changes:
----------
diff:
---
+++
@@ -8,3 +8,6 @@
## Add your entries here, make sure to append "target=dom0" to all allow/ask actions
+sys-gui @adminvm allow target=dom0
+sys-gui @tag:guivm-sys-gui allow target=dom0
+sys-gui sys-gui allow target=dom0
Total states run: 9
Total run time: 344.670 s
DOM0 configuration failed, not continuing
[Anirban@dom0 ~]$
My initial GPU-attaching attempts were “just attach the Renoir iGPU device”, and I could observe that:
qubes.skip_autostart, would make the system unusable (I’m being voluntarily vague here)qvm-prefs sys-usb does not reveal to sys-gui-gpu).Trying now to avoid possibly unsupported configuration by attaching the whole set of functions on the same device (which includes all USB controllers, which I detached from sys-usb for a first test)…
But then How to enable the (new) GUI VM? - #5 by marmarek talking about rollback does tell we have to unset autostart and default_guivm… so let’s set that and see… I see the system booting, but suddenly reseting, supposedly when trying to start sys-gui-gpu.
Naturally once you have set default_guivm to sys-gui-gpu, if you need to start with autostart disabled and need to start any app in a vm, you need to either default_guivm or the guivm pref for that vm back to dom0. Failing that, you can see the vm consuming CPU cycles… but no window on your screen.
Now I’m pretty sure I recently read in a post some xen and linux boot parameters to avoid the reboot and properly capture the dying words of the system, from @marmarek probably, but can’t find it, and can’t find the info in any “troubleshooting” page either – and just removing rhgb in the hope a cam will get anything useful is not much more than wishful thinking. Sure we could have the info in such a doc 
Edit: despite the difficulty, my cam can still capture part of a message from amdgpu claiming “Hotplug removal is not supported”, followed by a “finishing device” line.
Following this, the logical step seems to avoid attaching the iGPU to dom0 in the first place. Thus I’m extending the existing pci-stub config to include it (in addition to the dGPU): pci-stub.ids=1002:7340,1002:1636. To avoid any bad interaction, as above I detached the xHCI controllers from sys-usb and attached all functions to sys-gui-gpu.
Console on ttyUSB0 shows the following (pretty much obviously it gets interrupted when attaching to sys-gui-gpu):
[ 27.602043] pciback 0000:07:00.2: xen_pciback: seizing device
[ 27.602534] Already setup the GSI :36
[ 27.692618] pciback 0000:07:00.6: xen_pciback: seizing device
[ 27.693123] Already setup the GSI :36
[ 27.695599] xhci_hcd 0000:07:00.4: remove, state 4
[ 27.696021] usb usb4: USB disconnect, device number 1
[ 27.697030] xhci_hcd 0000:07:00.4: USB bus 4 deregistered
[ 27.697769] xhci_hcd 0000:07:00.4: remove, state 1
[ 27.698193] usb usb3: USB disconnect, device number 1
[ 27.698586] usb 3-3: USB disconnect, device number 2
[ 27.716608] xhci_hcd 0000:07:00.4: USB bus 3 deregistered
[ 27.718182] pciback 0000:07:00.4: xen_pciback: seizing device
[ 27.718680] Already setup the GSI :38
[ 27.764558] pciback 0000:07:00.5: xen_pciback: seizing device
[ 27.764697] Already setup the
… at which point I’m left with the lightdm screen showing but no possible interaction from USB devices or laptop keyboard (whose CAPSLOCK LED still responds) or touchpad (both usually still work with eg. sys-usb down).
If now I don’t forward the xHCI devices to sys-gui-gpu (but keep them in dom0, with the console going through them), I get called out for doing precisely this:
Sep 21 02:10:08 dom0 qvm-start[2427]: Start failed: internal error: Unable to reset PCI device 0000:07:00.5: internal error: Active 0000:07:00.4 devices on bus with 0000:07:00.5, not doing bus reset, see /var/log/libvirt/libxl/libxl-driver.log for details
… but said logfile does not have anything about the event, the above journalctl trace seems to be the only line about the event.
If I just stop using the USB console, a similar error happens. If I attach the xHCI controllers to sys-gui with all other functions of the iGPU PCI device, I’m back to a black screen (but the keyboard’s LED still answers), but don’t get any log on disk.
And using pci-stub to blacklist all ids for the functions in the device… leads to a similar-looking blackscreen, which this time gets a reboot after a few seconds.
Tough, not having a single usable debug channel
I’ve the same problem with , Q 4.1RC2- on Acer A515-52-56UN
no network working
dom0 $ qvm-start sys-net: Start failed: internal error: Unable to reset PCI device 0000:01:00.1: internal error: Active 0000:01:00.1 devices on 0000:01:00.1, not doing bus reset,
see /var/log/libvirt/libxl/libxl-driver.log for details
Thank you for advice howto fix it
please note that this thread is for sys-gui in qubes 4.1 alpha, which 4.1 alpha is completed it development. moreover, i don’t see your problem is related to the topic (is you trying to fix the usb mouse?)
you can create a topic asking this problem
This particular issue is addressed by attaching the PCI device with no-strict-reset=True. You may get some more inspiration in Salt recipes for easier switching sys-gui-gpu on/off.
Thank you all
The issue was whit sys-net. I fixed it
following PCI troubleshooting | Qubes OS PCI troubleshooting
You can also configure strict reset directly from the Qubes interface by following these steps:
Go to the sys-net VM settings
Go to Devices
Make sure the device is in the right field
Click “Configure strict reset for PCI devices”
Select the device, click OK and Aply
Thank you to the great Q Team