Restarting Windows during a Remote Desktop Session can prove to be way more difficult than it should since there the obvious methods are missing during a RDP session.

To shut down a remote computer when you are using Remote Desktop, press CTRL+ALT+END, and then choose whether you want to shutdown or reboot by using the red power icon in the bottom right corner.