I think I am using 3/8".
The diameter is really given by what you can hold, because modern ropes are ample strong even at smaller diameters.
I replaced one of the blocks for the main with this ratchet block (the one shown here is installed for the jib, but I use the same kind of block for the main).
I would not recommend the same kind of line as shown - it's too bulky, too heavy and to slippery (but it was free, and the perfect length when I needed a new jib sheet).
For my main I use FSE Robline's "racing sheet" which is smooth-running as well as grippy.
By using a ratchet block, I need to rely less on the mechanical advantage of the purchase if I want to hold the sheet in my hand (and not cleat it). My setup has 2.5:1 mechanical advantage (as measured at the end of the boom) which is equivalent to 5:1 for a pure center-boom setup. A 4:1 center-boom setup would also be typical, and many people have that. (My mainsheet setup is about the same as described here:
http://forum.daysailer.org/tech_rigguide.php).