I don’t want to install nvidia drivers 
I’ve fixed my issues by:
sudo qubes-dom0-update kernel
Then installed it via Yum-Extender Gui
Then according to “dnf list installed kernel” I had the old and new one still installed. I did:
sudo qubes-dom0-update kernel-qubes-vm --best --allowerasing
which is most likely not necessary since it is for the VMs not dom0.
Then:
sudo dnf remove kernel 5.4.98
However, I am not sure that it was the technically correct way. See also How to Update | Qubes OS.
If it was incorrect what I did, please comment on this post. You can never stop learning.