My theme (Adwaita) is a light theme, so this means that all the icons aren’t available.
My advice: merge the Restart - Shut Down - Lock Screen buttons into a sub-menu.
How : right-clic on Restart, Properties, Appearance: choose Session Menu, Actions: select Restart - Shutdown - Lock Screen, close. The new Session menu sub-menu displays the entries with the icon and the label, so you see Restart label.
Then you remove the now useless other buttons (Lock Screen and Shut Down, viewable in your screenshot) because they are in the Session Menu .
If you really want a Restart icon, then search in the XFCE project doc and forum (for XFCE 4.14), because IMHO it’s not a Qubes-OS related issue.