Are the users mainly anonymous or do they login and have specific content shown to them?
The difference comes from being able to cache content and serve that, if the content is mainly the same for all visitors. If it’s custom views for each visitor, then it’s a different answer than below.

When you say 2-3M uniques, do you mean unique visitors or page views. When talking about server capacity, you should only count page views, since that would give you the real numbers of load on the server.
If you meant page views, then it’s about 1 page view per second.

I would probably use Varnish (advanced cache), which connects to Nginx, then PHP-FPM 7.1 (which is more or less same speed as HHVM, but more actively developed), and database in MariaDB 10.1 (has some speed and feature advantages over MySQL and is not controlled by Oracle).
So it’ll look like this:

Visitor <-> Varnish <-> Nginx+PHP (WordPress) <-> MariaDB

Where Varnish, Nginx+PHP and MariaDB would be on their own separate server.

If you want extra fail-over options, you could take advantage of DigitalOcean’s new load balancer feature. It could be placed in front of any server, but I would probably just place it in front of multiple Varnish cache servers.

Visitor <-> Load Balancer <-> Varnish[2+] <-> Nginx+PHP (WordPress) <-> MariaDB

You need to measure the load of each of the services to figure out the exact capacity needed for each server. And if you need a load balancer in front of each server.

You’re saying, the web server is currently sometimes failing. What is failing? The entire server, Nginx, HHVM, MySQL or ?

@vitamincee

Setup really depends on what you’re currently running and how you’ve configured it to work. Before I could really recommend what I would transition in to, can you provide a little more information about your current setup?

1). Have you tweaked/tuned your current NGINX setup at all? If so, how?

2). Have you tweaked/tuned your current database server at all? If so, how?

3). Are you currently using any sort of caching – object, Memcached, Redis, etc?

4). What are your current server stats – CPU(s), RAM, etc?

5). Are you using HHVM specifically because of what it offers, or would you be open to PHP 7?

6). How many plugins do you have active? Any resource-intensive ones?

…

Examples of resource intensive plugins would be image optimizer plugins, such as Smusher and any similar plugins.

…

Beyond that, if there’s anything unique about your setup, please let me know.

@vitamincee

Thank you for the answers and thank you, @hansen, for the tag!

Before we can get to the part where we suggest an ideal setup for you to transition over to, the first thing I would recommend doing is finding out why PHP is crashing/failing. If you’re running a server that large and resources aren’t an issue, then sockets and TCP connections most likely are.

Stock OS deployments aren’t tuned for high-traffic scenarios. PHP-FPM defaults to socket mode on initial installation, though there’s an option to swap over to TCP connections. In either case, you may be approaching the limits of what a stock OS is capable of if connections begin to drop.

To identify any potential areas that may speak to why such failures occur, we need to check the log files and see what’s showing up.

The command we’d used to scan the logs and get the last x or xx lines would be tail followed by a -N where N is a number.

I’d recommend checking the syslog as well as the logs for both PHP and NGINX just to make sure we aren’t missing anything.

tail -50 /var/log/syslog

tail -50 /var/log/nginx/error.log

The PHP error log differs, so you’d need to see where it’s set to log and run the same on it.

…

I recommend checking your logs first so we can identify the why as moving to DigitalOcean won’t magically fix an issue where you’re hitting the walls of a stock OS that isn’t tuned for high-traffic.

If it does turn out to be that you’re hitting the wall, that can be tweaked accordingly, so that’s not an issue, but it’s a good idea to find that out now than it would be to move everything over and continue to run in to the same issues.

@vitamincee

If everything appears to be normal, there’s a few ways we can setup a cluster that would provide you with better reliability. Both are a little complex, and will take a little time to setup properly, and ideally, once setup, you need to work on optimizing NGINX, PHP-FPM, and MariaDB, otherwise moving to a setup like this isn’t going to provide much benefit at all.

NOTE: Since I don’t know your domain, I’ll be using domain.com in all examples.

…

Recommendation #01

My first recommendation is the more simple way of setting up a cluster that’ll allow you as the need arises, whether it’s vertically or horizontally.

What I would do is separate each concern. You’ve already got a jumpstart on this by offloading your database to its own server, so the concept is very similar.

Step 01:

Setup a load balancer. You can use DigitalOceans’ LB service, or setup NGINX as an LB that you have full control over. You’d point your DNS at this LB and it would handle traffic routing for you.

Step 02:

Setup NGINX + PHP-FPM (using PHP 7.1.x) on it’s own server, much like what you have now. We’d be moving away from HHVM to PHP 7.1.x in this case. HHVM is marginally slower in some areas and marginally faster in others, though ultimately, we’re looking for simplicity and consistency.

Step 03:

Setup Caching Servers, either Memcached or Redis. I’d pick which you’d like to use and setup 2x of them to start. The first one would be used to offload PHP Sessions so that we’re not storing them to files (which is the default) and the second would be for caching that’s handled by WordPress. When you start clustering, this is important as if you ever move to a multi-server setup to distribute content, you need session persistence and that’s not possible using file-based sessions.

Step 04:

Setup a MariaDB Server, much like what you’ve done already. I’d aim for the 10.x release. Right now, we’d setup a single database server. Since you’re not seeing extremely high loads right now, setting up a cluster or master-slave might be overkill.

Bring it Together

How this would work is something like this:

LB =>
   => NGINX/PHP-FPM => Caching Servers
   => MariaDB Server

The LB handles directing traffic to your NGINX/PHP-FPM instance. Once a visitor arrives, the caching servers come in to play and allow sessions to begin and cached data to resolve. Anything that isn’t in the cache, or can’t be pulled from cache would then hit the database server. On the database server, we’d need to implement some sort of caching as well (i.e. Query Caching) to speed things up there as well.

What this does is simply expand on what you already have and will allow you to increase complexity as needed.

Ideally, for the above setup, I’d recommend the following sizes (these being based off the resource information you’ve provided):

LB              => 2GB Droplet
NGINX/PHP-FPM   => 8GB Droplet
Caching Servers => 2GB Droplet x2
MariaDB Server  => 16GB Droplet

Now, the issue here is that 8GB Droplet for NGINX/PHP-FPM. You’ve stated that you’re using 50% of your current 480GB SDD HD, so the resolution to his would be to use DigitalOcean’s Block Storage option to attach a block device with XXX GB of additional storage. You would then configure WP to use that device, once mounted, for your storage.

WordPress does allow you to change the ./wp-content location/URL, and that’d be set in the ./wp-config.php file, so that’s how we’d manage changing the content locations.

Since you’re only using ~5% of the Web Servers RAM (32GB), which is about 1.6GB, an 8GB Droplet should be fine for it. Since you’re using ~25% of the Database Servers RAM (24GB), which is about 6GB, I’d definitely shoot for the 16GB Droplet as that gives you over double usage and allows room for increased RAM usage from optimization and properly using RAM.

That brings the total monthly cost for the above to around $280 per month. Of course, this is a very basic setup, but we can scale it from there as needed should you find that it’s not coping.

Other Notes

The above setup should provide a better working environment, but optimization is going to be key here, just as much as proper setup is (i.e. firewall, security, etc). This is what I would classify as a starter setup that gets you working more with cluster setups.

You need to make sure NGINX, PHP-FPM, MariaDB and your caching servers are optimized for the work load, otherwise you’ll end up in the same situation.

…

I’ve setup clusters like these for multiple private clients over the years and they work well, while providing room to grow. They can be a bit complex at first glance, but they do provide better flow and handle traffic far better than a simple two-server setup. Yes, it’s more to manage, though when it comes to scale, these options make such far easier and much less of a headache.

I know this is old but on the off-chance you’re still dealing with any issues like this or for anyone else reading it… the recommendations here are a bit overkill. I have a server in place with your exact specs (except a slightly smaller SSD) and with an actual caching mechanism in place, none of it is necessary.

I also get nearly your exact # of pageviews between three websites and even my own setup is overkill. I don’t need a server with this much power for my needs but I have it anyway for peace of mind.

Setup your stack (but use php 7.x - preferably 7.0 or 7.1), install Opcache, Memcached, and ACPu. Give OpCache to PHP, Memcached to the database, and ACPu to object cache (assuming you’re using W3TC) and Disk Cache do “enhanced”. I also minify using W3TC and give that to Memcached, too.

All that said… and all of my tinkering… nothing I’ve ever tried could beat Varnish. Using Varnish I’ve seen traffic spikes of 5000+ concurrent users and the server load barely touch 1. The reason I stopped using it was I couldn’t quite figure out how to get it to properly create and serve different caching for desktop and mobile (which is something I require for my very specific ad layout). Every configuration I tinkered with would cause one issue or another… so I had to go back to the drawing board.