One big droplet or multiple small droplets?

  • Posted on August 6, 2014
  • komsit37Asked by komsit37

I am new to network architecture. Would like to hear from expert. I am running web server, crawler, and database. should i get a big droplet (big CPU, memory) and put all of them in one droplet, or just get 3 small droplets and run them separately?

Some more details of my nodes:

  • crawler - adding 10-20 Mb data per day, runs 3-4 hours a day. it’s quite heavy when running since it’s phantomjs/java
  • web server - mostly analytics stuff and data visualization (running R)
  • database - column-based, in-memory database, 100-200Mb data so far

The architecture should support scaling 10-100 times bigger in the long run

Now i have 1 small node for crawler and database (512Mb ram) and 1 mid-size node for web server (1Gb ram) since some R packages required more than 512 Mb to build

Submit an answer

This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

Sign In or Sign Up to Answer

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

Want to learn more? Join the DigitalOcean Community!

Join our DigitalOcean community of over a million developers for free! Get help and share knowledge in Q&A, subscribe to topics of interest, and get courses and tools that will help you grow as a developer and scale your project or business.

I run two droplets - one as a mail server and the other as a web server. I started out with just one, but quickly learned that SpamAssassin and ClamAV consume quite a bit of memory on their own, and serving a couple of web sites and Ruby-based web applications through Apache didn’t make it any better. There was a lot of memory swapping, and a lot of e-mail wasn’t getting filtered because it was taking too long. A single 2 GB droplet would probably do the job, but I save a little bit of money by running my mail server on a 1 GB droplet, and NginX on a 512 MB droplet.

There’s no single “correct” answer for a question like this, but you current architecture sounds like a fine approach to me. Though I might move the database on to the larger droplet with the web server. Is this stuff all for personal/internal use? That would make a difference. If it’s not, you don’t necessarily need to think too much about different ways to scale up to handle increases in traffic. My main concern would be the Java based crawler competing for resources with the other services.