I downloaded the panning city youtube video in different resolutions and tried to play it on modern CPU with a lot of VCPUs, 16GiB+ RAM in mpv/smplayer/mplayer with different options.
And in all cases the video output lags during panning and other moments.
That’s why I think the issue is in Xen+Xorg interaction. I believe CPU is able to decode the stream on fly for sure.
=> So, the problem probably lies in Xen/Xorg/QubesOS not good video output architecture.