I found a setting change in VLC that improves a lot decoding, but as it is done 100% software it will always be inefficient anyway: VLC video playback CPU usage improvement
However with this VLC change my i7-1260P is able to decode a 1080p x265 video with a single core at 30% load, which is not that bad.