One big droplet or multiple small droplets?

August 6, 2014 2.1k views

I am new to network architecture. Would like to hear from expert.
I am running web server, crawler, and database. should i get a big droplet (big CPU, memory) and put all of them in one droplet, or just get 3 small droplets and run them separately?

Some more details of my nodes:

  • crawler - adding 10-20 Mb data per day, runs 3-4 hours a day. it's quite heavy when running since it's phantomjs/java
  • web server - mostly analytics stuff and data visualization (running R)
  • database - column-based, in-memory database, 100-200Mb data so far

The architecture should support scaling 10-100 times bigger in the long run

Now i have 1 small node for crawler and database (512Mb ram) and 1 mid-size node for web server (1Gb ram) since some R packages required more than 512 Mb to build

2 Answers

There's no single "correct" answer for a question like this, but you current architecture sounds like a fine approach to me. Though I might move the database on to the larger droplet with the web server. Is this stuff all for personal/internal use? That would make a difference. If it's not, you don't necessarily need to think too much about different ways to scale up to handle increases in traffic. My main concern would be the Java based crawler competing for resources with the other services.

I run two droplets - one as a mail server and the other as a web server. I started out with just one, but quickly learned that SpamAssassin and ClamAV consume quite a bit of memory on their own, and serving a couple of web sites and Ruby-based web applications through Apache didn't make it any better. There was a lot of memory swapping, and a lot of e-mail wasn't getting filtered because it was taking too long. A single 2 GB droplet would probably do the job, but I save a little bit of money by running my mail server on a 1 GB droplet, and NginX on a 512 MB droplet.

Have another answer? Share your knowledge.