jeroennn
By:
jeroennn

What are the requirements for successful health check?

February 21, 2017 1.7k views
Load Balancing Ubuntu 16.04

Can we run a script on the webservers that returns HTTP 200 if the server is properly configured, and HTTP 5xx if something is wrong, so the loadbalancer will take it out?

2 Answers

This is written in the official tutorial:

Health Checks ensure that Droplets are available. By default, they test endpoints every 10 seconds. The health check for our forwarding rule will ping each Droplet’s web server on port 80, and if the server fails to respond after three tries, it will be removed from rotation. The Load Balancer will continue to ping the server, and once it has successfully received a response 5 consecutive times, the server will be returned to the pool.

So I would guess that's kind of a no, but you can run a cronjob script every minute that will close your port 80 or remove the server from the LB group or something else. It's not perfect, but maybe you should suggest it on UserVoice: https://digitalocean.uservoice.com/forums/136585-digitalocean/

edited by asb
  • I'm a bit confused, what is the difference between TCP health checks and HTTP health checks in that case?

    • Only replying as this comes up as the top result looking for digitalocean loadbalancer health check and others may find it useful.

      As far as I'm aware TCP checks the port accepts a connection only, whereas HTTP looks for a 200 response I believe. I've only ever used the HTTP check.

      We've used the following code as our health-check.php originally.

      <?php
        $config = [
           "server"     => "172.x.x.x:3306",
           "username"   => "healthcheck",
           "password"   => "password"
        ];
        header('Content-type: application/json');
      
        $link = new mysqli($config['server'],$config['username'],$config['password']);
        $mount = is_readable("/mnt/www/index.php");
      
        if (($link->connect_errno) || (!$mount)) {
          http_response_code(503);
          echo('{ "status": "Unable to connect" }');
        }
        else {
          http_response_code(200);
          echo('{ "status": "Connection successful." }');
        }
      
      ?>
      

      I'm not entirely sure if I found the above code online or had to build it up from different bits, if this is someone else's I'm happy to give credit.

      It basically makes sure that the web server gets no errors when connecting to the database server and the filesystem mount is working. If either fails the loadbalancer takes that server out of action.
      WARNING: If you put this on all servers and the DB or Filesystem goes down then ALL servers will be taken from the loadbalancer. You get a rather white page with something like 'unable to handle request' which isn't good for visitors.

      To deal with the issues in the warning we replaced the above with:

      <?php
      function Test_Database() {
      
        $config = [
           "server"     => "172.x.x.x:3306",
           "username"   => "healthcheck",
           "password"   => "password"
        ];
      
        $link = new mysqli($config['server'],$config['username'],$config['password']);
      
        return $link->connect_errorno;
      }
      
      function Test_Mount() {
      
        $mount = is_readable("/mnt/www/index.php");
      
        return $mount;
      }
      
      function Test_UpTime() {
        $str   = @file_get_contents('/proc/uptime');
        $num   = floatval($str);
      
        return $num;
      }
      
      function Always_Online_Server() {
      
        $hostname = gethostname();
      
        if ( $hostname == "SERVER01" ) {
          return true;
        } else {
          return false;
        }
      }
      
        $db = Test_Database();
        $mount = Test_Mount();
        $aos = Always_Online_Server();
        $uptime = Test_UpTime();
      
        header('Content-type: application/json');
      
        if ( $aos ) {
          http_response_code(200);
          echo('{ "status": "Connection successful." }');
        } else {
          if (($db) || (!$mount) || ($uptime < 300)) {
            http_response_code(503);
            echo('{ "status": "Unable to connect" }');
          }
          else {
            http_response_code(200);
            echo('{ "status": "Connection successful." }');
          }
        }
      ?>
      

      I'll admit it's not the nicest code :) but this would leave 1 server as always reporting OK to the loadbalancer regardless of the database and mount. That way if wordpress has a database connection problem you can customise that error page. The server stills fail and is taken away from the loadbalancer if the web or php services are down. There's also the added check to make sure the servers have been up for more than 5 mins to allow new servers some time to settle before reporting OK to the loadbalancer.

      I do hope this helps someone.

Have another answer? Share your knowledge.