Question

HTTP Health check using DigitalOcean Load Balancer & ServerPilot

Posted July 31, 2018 2.5k views
Load BalancingUbuntu 16.04

Hi,

This is not a question, but a something that I had to share with anyone who may come across the same issue.

I have setup a Highly Available (HA) WordPress environment using 4 droplets. Two are application droplets managed by ServerPilot and the other 2 are MySQL 8.x Database servers with Group Replication, load balanced with ProxySQL (highly recommend this).

What I wanted to do was use the DO Load Balancer (LB) service for the 2 application droplets but I could not get the HTTP Health check working. The Load balancer kept on complaining that the droplets were offline, when they were not. The TCP health check was working, but I needed HTTP.

Long story short, there was no Default app for the application settings in ServerPilot, and “Deny requests for unknown domains” was ON. Since the DO Load Balancer uses a the private IPs to check the health of each droplet and not a domain, the health checks were failing (Nginx was returning a 444). Once the “Deny requests for unknown domains” was turned OFF, the LB was able to get a 200 from Nginx and everyone’s happy.

Last but not least, make sure you have assigned the droplet private IPs to your applications in ServerPilot to make sure they are accessible by the LB for health checks

Hope that helps someone.

A

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
2 answers

We had a similar problem with a Laravel application that we had built.

Our Laravel application was using “domain routing”, which meant that any request that was not for a domain in our list was returning a 404 error. The load balancer is sending the domain name “cloud.digitalocean.com”, which our application did not recognize.

Once we added routing to recognize this domain name, the HTTP health checks began to work fine.

For those that land on this from google, I had to sort out the same thing with Django today.

Problem is the DigitialOcean load balancers do not allow you to shape the traffic for health checks. You can re-implement ALLOWED_HOSTS from the Django config at the webserver level by only allowing traffic addressed to your domain, like so for example in Apache…

<Directory /var/wsgi/mysite/myapp/>
        <Files wsgi.py>
            Require ip 10
        Require host cloud.digitalocean.com
        </Files>
    </Directory>

And then just set Django ALLOWED_HOSTS = [’*’] but that leaves a bad taste…

On second thought, all traffic coming through the load balancer is going to be from a 10.x IP range so all traffic will pass with this rule. However, this rule does restrict people from connecting to your site via your droplet’s public IP, so leave it in place, because you want all traffic to pass through the load balancer, not go to the droplet directly.

You can add more rules to the same directory below that restriction…

<Directory /var/wsgi/mysite/myapp/>
        <Files wsgi.py>
            Require ip 10
        Require host cloud.digitalocean.com
        </Files>
    </Directory>
    <Directory /var/wsgi/mysite/myapp/>
    SetEnvIfNoCase Host www.mysite\.com SITE_HOST
        Order Deny,Allow
    Deny from All
    Allow from env=SITE_HOST
    </Directory>

This effectively fixes the droplet’s webserver, but WAIT, the load balancer will get caught by this rule because you can’t tell it to check your domain. If you want the load balancer to eval the health of your web app, it’s going to get 403’d and turn itself off because it checks by IP, you can’t tell it to check by name.

Also, you can’t eval multiple statements with a single SetEnvIf. So here’s the final fix…

<Directory /var/wsgi/mysite/myapp/>
        <Files wsgi.py>
            Require ip 10
        Require host cloud.digitalocean.com
        </Files>
    </Directory>
    <Directory /var/wsgi/mysite/myapp/>
    SetEnvIfNoCase Host www.mysite\.com SITE_HOST
         SetEnvIfNoCase Host cloud.digitalocean\.com SITE_HOST
        Order Deny,Allow
    Deny from All
    Allow from env=SITE_HOST
    </Directory>

Using two statements effectively accomplishes “OR”. If the host requested is not mysite.com it goes to the second statement. If the host requested is not cloud.digitalocean.com in the second try, then the variable gets set again. From there all hosts are denied, except the one in the variable, which by virtue of making it that far down into your conditions, has failed both of your checks.

Now you can safely go set ALLOWED_HOSTS = [’*’] because you have effectively accomplished the same functionality in your webserver, and don’t need django to do it for you because invalid hosts won’t make it to django.

Submit an Answer