I am wondering if the output signal to the monitor has been encoded or is it native signal. This always seems to be much higher quality than the signal on youtube.
The HDMI output displays the hardware encoded video and the user interface. The quality of the video on the stream itself is entirely dependent on your preferences for encoding (such as frame size and bitrate) as well as the strength and speed of the network itself.
If you have a fast and high upload bandwidth on your network, you can increase the bitrate to a higher one to give you better quality for your stream. If you are using Wifi, an Ethernet connection will provide a more stable network connection for streaming