47270b9a9b9713bc421e5f39e7cbb037095a126c
By:
weblora

Mail Server Stops Working When Switching To Cloudflare DNS

January 5, 2017 296 views
Email Apache DNS System Tools Ubuntu 16.04

Upon switching over to Cloudflare, the mail server that I have set up with PostFix and Dovecot stops working. By "stops working", I mean that I am no longer able to send or receive mail from my Thunderbird client. As soon as I switch back to Digital Ocean's nameservers, I am able to receive the mails that had queued up while on Cloudflare's NS.

So the mailserver doesn't stop working entirely, however when I try to authenticate/login to my mail account through Thunderbird, to send/receive, the connection just times out/server doesn't respond.

My mailserver has DKIM and SPF records in use.

Any help would be much appreciated!

1 comment
  • Did you lower your TTLs prior to the inital DNS changeover to CF? When you next try, confirm the following after DNS propagation:

    Check MX

    $ dig weblora.com mx +short
    0 mail.weblora.com.
    

    Check MX Answer

    $ dig mail.weblora.com +short 
    178.62.61.111
    

    The above are your current settings and these both should be confirmed after nameserver update for your domain.

    If you still can not connect via Thunderbird, test your MTA with Telnet

    telnet mail.weblora.com 25
    

    Regarrrrrrrds... not A pyrate ;0

4 Answers

@weblora

CloudFlare only resolves the DNS entries that you provide, so if you're not receiving e-mail after switching to their DNS, the issue would most likely be due to misconfiguration.

The DNS that shows here at DigitalOcean should be identical to that which is setup for CloudFlare (i.e. all A, CNAME, MX, TXT, etc). You also have to modify your domains DNS to use CloudFlare's provided DNS servers after your setup.

If you can, please provide links to a screenshot of your DNS setup with DigitalOcean as well as a screenshot of what shows up for the same domain in CloudFlare's CP. I'll be more than happy to take a look at it for you.

  • @jtittle

    Thank you so much!

    Here are the current DNS setups: http://imgur.com/a/fkdqu

    (My registrar's DNS settings have been set to CloudFlare)

    • @weblora

      Thanks for the screenshots!

      The first issue I can see, now that I know the IP/Domain, is that there's not a valid Reverse DNS (i.e. PTR Record) entry for mail.weblora.com. Why does this matter?

      Most mail servers will reject e-mail sent from a web server that does not have a valid PTR record. They may query for a period of time to see if one exists, though ultimately, the mail will be rejected. Locally, the mail may never even leave the server, so it's actually a good thing that this happened as it gives you time to fix this before it becomes an issue when you're really relying on e-mail to come through.

      So what can you do?

      With DigitalOcean, you're only provided with one Dedicated IP, so it's automatically assigned a PTR record. The downside here is that since you don't have multiple dedicated IP's, you need to name your Droplet what you intend on using for your mail server.

      In this case, it'd be mail.weblora.com. You also need to set the hostname via SSH using:

      hostname mail.weblora.com
      

      You can then verify the change by simply using:

      hostname
      

      Once this is setup through the DigitalOcean CP and on your Droplet (both need to be done). You can verify the PTR record at DigitalOcean by:

      1). Login to the DigitalOcean Control Panel
      2). Click on Networking
      3). Click on PTR Records

      You should see the IP of your Droplet with a PTR record for mail.weblora.com.

      You already have an A entry pointing the sub-domain to the correct IP, so nothing more needs to be done there and the PTR record does not need to be added to CloudFlare (it only needs to be properly setup on DigitalOcean's side, which the above will do).

      Once you have a valid PTR pointing to mail.weblora.com, everything else should be in valid, working condition. From there, any issues that arise or prevent you from connecting would need to be brought to the attention of CloudFlare's support team to see if they are potentially blocking the IP or a subnet (possible), if it's on a blacklist, etc.

      ======

      That being said, ideally, web and mail servers should not be hosted on the same Droplet, more so because of the above requirement. Ideally, you want to setup one server as mail.weblora.com and then another as web.weblora.com. This separates concerns and isolates your mail server from that of your web server.

      • @weblora

        To change the name of your Droplet here at DigitalOcean, view the list of your Droplets, click on the one you want to rename and you should be brought to your main page with all your graphs.

        Your hostname should be at the top. Click on it and an edit field will pop up. Simply change the name and click the blue pencil icon to the right to save the name.

        After you do this, you can set the hostname via SSH as noted above.

      • Thanks for the great reply! I appreciate it far more than I could possibly begin to articulate.

        So I tried the first bit (changing the hostname and DO droplet name, however, the problem unfortunately persisted), so I am thinking to go with your ideal description at the bottom of your response.

        I am however having difficulty wrapping my head around how to run two separate servers, for the same domain.

        I have set the DNS of my registrar back to that of Digital Ocean, to try and get it sorted out on the DO side first.

        I am going to wait for the DNS changes to fully propagate, then I will start a second droplet to handle the mail for Weblora, is it alright if I come back to you when I inevitably run into a wall, again?

        Kindest Regards,
        Jay.

        • @weblora

          No problem, always happy to help!

          Essentially, what you're going to do is create two droplets and name one of them mail.weblora.com and the other web.weblora.com.

          You can change web to anything you want, but keep mail as the other.

          Once the two droplets are live, login to each and make sure the hostname is set by running the hostname command on each one. If each droplet returns the full hostname of the droplet, you're ready to proceed. If they don't, set the hostnames by running the below (once on each server):

          hostname mail.weblora.com
          

          and

          hostname web.weblora.com
          

          Now, modify your DigitalOcean DNS and make sure each Droplet has a valid A entry pointing to their respective Public IP Addresses.

          Before proceeding further, delete your domain from CloudFlare. It's best to start fresh and let CloudFlare re-scan your Domain and its DNS.

          Now we have two Droplets ready to configure, so start with the mail Droplet. I don't know how you configured your current Droplet. I didn't see a mail server one-click, so I'm guessing you used another script? If so, simply re-run that script on the new mail droplet and let it run it's course. Since the existing one is working, I'm assuming that won't be an issue and that you won't run in to any issues. If you do, just ask for help :-).

          Once the mail droplet is ready to go, we can move on to the web droplet.

          How we configure this droplet really depends on what you need. What kind of website are you trying to host? (i.e WordPress, Ghost, etc) -- that will determine what we need to do.

          How this all ties together is simple. You have an @ entry in your DNS. This entry is the one you point at the IP address of the web server. So your main domain will point to the same IP as web.weblora.com -- this is how, once a request is made, routing understands to send requests for weblora.com to the IP of the web.weblora.com droplet and mail to mail.weblora.com.

          So your DNS will look something like this:

          web        A          WEB_DROPLET_IP
          mail       A          MAIL_DROPLET_IP
          @          A          WEB_DROPLET_IP
          www        CNAME      weblora.com
          
          • Thank you SO much! You are absolutely incredible!

            Does this look right to you? http://i.imgur.com/nbkV1pS.png

            I see that I get some errors if I do a DNS Check on http://mxtoolbox.com/

            I was wondering... would it make sense for me to use the firewall on my mailserver to disable port 80 (http) and 443 (https)? Since I will only be using the server for mail handling?

            Also, what is the best way to mitigate the risk of a DDoS attack? The biggest part of me wanting to use CloudFlare for the web server was to mitigate this risk, but since the origin IP of my mail server is visible, I was wondering if there is anything I can do about that?

            Thanks again! :)
            Jay

make sure you have all the DNS records same on cloudflare as Digitalocean. and those related with your emails should be gray clouded.

@weblora

Starting a new reply as there's not an option to reply to your last comment.

So, as far as your DNS at DigitalOcean, as long as those are the correct IP's, it looks perfect! I wouldn't foresee any issues once you run CloudFlare to scan the domain.

As far as the errors on your MX lookup, the first one concerning DMARC is something you can handle by configuring DMARC through your mail configuration, though it's not a huge concern for the initial setup or getting things working. Some providers are checking DMARC, though I've not ran in to any issues with providers rejecting e-mail on domains that do not have it setup (at least currently).

The HTTP error showing up simply means that either your web server isn't setup to receive requests (i.e. NGINX or Apache isn't serving requests) or there was a failure on initial request. As long as you're able to access your domain and view the page once your server is setup, that one can be overlooked for now.

The third error is a blacklist error on the mail servers IP. This happens as the IP's that DigitalOcean assigns are used by many users before they become yours. It's possible that DigitalOcean had a spammer come on board, send boatloads of e-mails, and the result was that the IP was blacklisted at one of the spam databases.

There's good and bad news with the blacklist. The good is that it's not one of the major spam databases, so you're not likely to see mail bounce as a result. The bad, even if you do, they don't offer a way to remove the listing just yet.

You can keep at eye on:

http://psky.me

.. which will allow you to keep up with when you can apply to remove the IP. Essentially, once able, you'd get in touch and let them know that you are not the previous owner, are running a new mail server and it's for personal mail (or something similar). They'll check the IP against their records and if they find the submission to be valid, they will remove it.

  • @weblora

    RE: DMARC

    I'd recommend reading over the post below. This is a really good post covering a lot of detailed configuration and setup on Postfix, including SPF, DKIM, and DMARC (all things that are complex, but are going to make life a little easier, once they are active).

    https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/

    Take your time reading through it. Don't rush it. It's easy to make mistakes, trust me, I've made plenty in the past :-).

    • You are a saint!

      Could you by any chance point me in the right direction in terms of getting my mail.weblora.com to send mails to web.weblora.com clients (or at least getting my web.weblora.com server to pretend it is web.weblora.com)?

      Let's say a user forgets their password and prompts recovery on web.weblora.com, how would I get the recovery email to them, from the mail.weblora.com IP address? As I don't want to expose my web.weblora.com origin IP?

      I am using PHP's mail() to send the mails. :)

      I have Postfix/Dovecot running on my web.weblora.com & mail.weblora.com server.

@weblora

All public web servers expose their public IP addresses. The only IP that is hidden from public access is the private network IP. Anyone could get the IP of a web server by performing a DNS lookup. That said, there's no need to worry about public IP exposure -- properly setting up your firewall will mitigate most common concerns regarding public exposure.

Beyond the firewall, further locking down the droplet and tightening security will further prevent the majority of other cases.

Sending Mail w/ PHP

As far as sending e-mail using PHP, the mail() function really doesn't accept connection parameters on the fly so you wouldn't be connecting to your mail server using this function.

The following lines of code should send an e-mail without any issues:

<?php
$headers    =   'From: user@weblora.com' . "\r\n" .
                'Reply-To: user@weblora.com' . "\r\n" .
                'X-Mailer: PHP/' . phpversion();

/**
 * 1). E-Mail Recipient
 * 2). E-Mail Subject
 * 3). E-Mail Message
 * 4). E-Mail Headers
 */
mail(
    'webmaster@weblora.com',
    'Testing An E-Mail Using PHP 7',
    'This is a test message using PHP 7.x to send.',
    $headers
);

... of course, since you're not connecting to your mail server, this e-mail will most likely end up in the SPAM box of whatever provider it's being sent to since there's absolutely zero authentication using the above.

If you want something more advanced, such as actually sending through your mail server, I'd recommend using something such as PHPMailer:

https://github.com/PHPMailer/PHPMailer

or SwiftMailer

https://github.com/swiftmailer/swiftmailer

These libraries provide a feature-rich set of functions that'll allow you to do what you need to do and connect to your mail server by passing the correct details during configuration.

If you're using a CMS, such as WordPress, you can use a plugin to handle the connection for you. You can take a look at:

https://wordpress.org/plugins-wp/wp-mail-smtp/

Have another answer? Share your knowledge.