This is is going to be short and sweet. If you are running opensmtpd or dovecot inside FreeBSD jails you may need specific changes for this configuration, depending on your jail setup.
For dovecot I had to add
listen = * to the config.
For opensmtpd I needed to have
listen on localhost port <portnumber> instead of
listen on lo0 port <portnumber> for smtpd.conf.