10 separate droplets w 10 different IP adds or one droplet w 10 parallel tasks?

January 10, 2015 1.2k views


I need to scrape many hundreds of thousands of URLs. Should I go for 10 separate droplets for 10 parallel url scrapers, or should I go with 10 parallel scarpers within the same droplet?
I suspect that IP addresses need to be different for the first case (10 different droplets). Is that correct?


2 Answers

If you can afford the cost, and the system won't fall under "abuse", I'd say go with the 10 droplets plan. This will give you more CPU than a single droplet.

As for a single IP, you can set up a SSH tunnel from 9 of the servers to the 10th. This will give you the same IP across all requests, even though they are coming from different servers.

  • Thanks JonsJava.

    One more q: Can I have different IP addresses, one for each droplet?

  • Each droplet actually gets its own individual IP, so, yes.

    Think of a droplet as a computer. Each one is a completely stand-alone system, separate from all others (unless you join them in some way, which is more technical that I'm going to go right now).

Have another answer? Share your knowledge.