"Do I need to create something like mail.example.com?" Yes.

"Also once the installation is done, what DNS records should I add for the example.com domain so that email would work?"

If you're using DigitalOcean's DNS Manager, you would need to add the following MX record: 1 - mail.example.com. (don't forget the trailing dot).

Thank you for the quick response.

So now I:
- Installed iredmail on a separate droplet via guide.
- Created mail.example.com pointing to that droplet.
- Used mail.example.com as the first virtual domain name during installation.
- Added a mail.example.com. MX record with priority 1 in the example.com DNS settings.
- Went to /iredmailadmin post installation, and added example.com to the list of domains. (Now I have mail.example.com and example.com in iRedAdmin settings.)
- Added a new user@example.com in iredadmin settings.

Now whenever I log into roundcube using user@example.com, and try to send email to someone@email.com, I get:

SMTP Error (451): Failed to add recipient "someone@email.com" (4.3.5 Server configuration problem).

I also cannot receive any mail.

Help? Once I get a hang of it, I should be able to figure everything else out.

To make sure you configured your DNS recoreds correctly, run the Check Domain Configuration tool.

Thanks!

The configuration tool returns this:

The mail server for mail.example.com accepts mail for mailbox 'postmaster@mail.example.com'.

It also passes everything else. No red flags.

I still get this when I try to send email:

"SMTP Error (451): Failed to add recipient "someone@email.com" (4.3.5 Server configuration problem)."

And any incoming email to the mailbox also does not get received.

I tried looking all over the internet, but found no solution to the problem.

Do you know what might be causing that?

Okay, so the droplet reboot fixed the 4.3.5 server configuration error, and sending mail works now. But I cannot receive any mail. Possible reasons?

Final Update:

Seems like running iptables -F, and rebooting the droplet again fixed the "not receiving email" issue as well.

Thanks for all the help! :)

"But I cannot receive any mail."

How do you know? Do the emails get bounced back or do they simply disappear? Is there anything in the error logs?

If you have a firewall, make sure to open up the relevant ports.