SMTP in Registration mail VS Contact page

I've got a strange problem that I've had since my first installation and never solved BUT now need to fix.

I have my site set to use the email verification system for registrations and all the notification emails are being received. Things are going fine there.

But I don't get emails from my "contact.php" contact page. There is no error. It says the email is sent, but I never get it.

Here's the strange part:

I currently have my smtp host set in my main settings (miscellaneous section).

If I REMOVE the smtp server from the settings, I get the contact emails, but the registration emails don't get sent out.

I've read some threads about other problems with the contact page or reg emails, but nothing seems to address this problem.

Any ideas?