You might think of this as a core thing with somewhere, perhaps in settings, that you define SMTP server, port, username and password. Apparently no.
It would appear that at the moment, v3.5, WordPress uses PHP mail. I am assuming that most people, by far, use a hosted solution so mail can be done for them.
In my case I have WordPress on a server behind my DSL connection so I need email setup to suit my ISP and so on.
Resolution
It seems that now the easiest way is via a plugin that changes WordPress to use SMTP. In my case I chose “Configure SMTP”.
As far as I can tell it works great I can give it the details of my ISPs SMTP server with my specific username and password. Whats more I am able to configure the “reply” email address and “reply” email name, something else that doesn’t appear easily configurable.