How to setup global load balancing using Digital Ocean DNS, Ubuntu, and Nginx?

Hi folks, I’m currently experimenting with methodologies on how to best implement a global load-balancing layer for my app servers on Digital Ocean and there’s a few pieces I’ve yet to put together.

The Goal Offer highly-available service to my users by routing all connections to the closest ‘cluster’ of app or api servers in SFO, NYC, LON, and eventually Singapore.

The Stack

1. Ubuntu 14.04
2. Nginx 1.4.6
3. node.js
4. MongoDB from Compose.com (formerly MongoHQ)

Current Working Methodology A (minimum) trio of global-balancing servers receive all traffic. These servers are DNS Round-Robin balanced as illustrated in this (frankly confusing) article: How To Configure DNS Round-Robin Load Balancing.

Using the Nginx GeoIP Module and MaxMind GeoIP Data the origin of any given request is determined down to the $geoip_city_continent_code.

The global-balancing layer then routes the request to theleast connected server on the load-balancing layer of the appropriate cluster: nyc-load-balancing-1, sfo-load-balancing-3, lon-load-balancing-2, etc… This layer is also a (minimum) trio of droplets.

The regional load-balancing layer then routes the request to the least connected server in the app or api layer: nyc-app-2, sfo-api-1, lon-api-3, etc…

The details of the Nginx kung-fu can be found in this tutorial: Villiage Idiot: Setting up Nginx with GSLB/Reverse Proxy on AWS. It’s actually very cool.

More general info about Nginx load-balancing is available here and here.

Global Domain Breakdown

So once I rig everything up, my DNS would look something like this:

**GLOBAL**
global-balancing-1.myapp.com
global-balancing-2.myapp.com
global-balancing-3.myapp.com

**NYC**
nyc-load-balancing-1.myapp.com
nyc-load-balancing-2.myapp.com
nyc-load-balancing-3.myapp.com

nyc-app-1.myapp.com
nyc-app-2.myapp.com
nyc-app-3.myapp.com

nyc-api-1.myapp.com
nyc-api-2.myapp.com
nyc-api-3.myapp.com

**SFO**
sfo-load-balancing-1.myapp.com
sfo-load-balancing-2.myapp.com
sfo-load-balancing-3.myapp.com

sfo-app-1.myapp.com
sfo-app-2.myapp.com
sfo-app-3.myapp.com

sfo-api-1.myapp.com
sfo-api-2.myapp.com
sfo-api-3.myapp.com

**LON**
lon-load-balancing-1.myapp.com
lon-load-balancing-2.myapp.com
lon-load-balancing-3.myapp.com

lon-app-1.myapp.com
lon-app-2.myapp.com
lon-app-3.myapp.com

lon-api-1.myapp.com
lon-api-2.myapp.com
lon-api-3.myapp.com

And then if there’s any strain on any given layer, in any given region, I can just spin up a new droplet to help out: nyc-app-4.myapp.com, lon-load-balancing-5.myapp.com, etc… And thanks to Digital Ocean, our new benevolent supreme SaaS, this is quick and cheap to do.

Questions

Where do a I put the global-balancing servers? It strikes me as odd that I would put them either all in one place, or spread that layer out around the globe either.

Say, for instance, I put them all in NYC. Then someone from France hits my domain. The request would go from France, to NYC, and then be routed back to LON.

Or if I put one of each in SFO, NYC, and LON then isn’t it still possible that a user from Toronto (Parkdale, represent) could send a request that ends up going to LON only to be routed back to NYC?

Do subsequent requests get routed to the same IP? As in, if a user from Toronto sends a request that the global-balancing layer determines should be going to NYC, does the next request from that origin go directly to NYC, or is it still luck of the draw that it will hit the nearest global-balancing server (NYC in this case).

What about sessions? I’ve configured Nginx to use the ip_hash; directive so it will direct the user to the same app or api endpoint (a node process, in my case) but how will global balancing affect this, if at all?

Any DNS Examples? I’m not exactly a DNS expert (I’m currently trying to figure out why my CNAME records aren’t resolving) but I’m a quick study when provided with a solid example. Has anyone gone through this process before and can provide a sample of what the DNS records look like for a successful setup?

What about SSL/TLS? Would I need a certificate for every server, or just for the three global-balancing servers since that’s the only public-facing gateway?

If you read this whole thing then reward yourself with a cupcake. Thanks in advance for any help.