Is Load balancing limited at 1000 requests per second?

April 13, 2017 4.5k views
Load Balancing Ubuntu 16.04
yenthanh132
By:
yenthanh132

Hi everyone,

Currently I want to scale my API servers, so I create some additional droplets and try to use the Load Balancing feature for those droplets. When I try to test the performance with loader.io (Sending 5k requests per second) I realize that the maximum number of success request/s is exactly 1000 (and a lot of timeout requests). Then I try to create another droplet as a load balancer using nginx and do the same test. The result shows that there are over 3000 request/s (and just a little timeout requests). So I guess that Load Balancing is limited at 1000 request/s. Is that true? Can I do anything to lift this limit?

3 comments
  • thienohs March 23, 2018

    @yenthanh132 @timothyleland @shatrughanjha
    Hi guys, did you guys figure out the problem with Load Balancer ?
    I did a test with loader.io and it seems to work with more than 5000 requests per second, there are timed out about 33%. So it didn't seem to fit the description of @yenthanh132 question.

    Not sure are you still able to reproduce the issue with Load Balancer limit of 1000 requests/per second ? I'm about to roll out a distributed back-end and wonder whether should use Load Balancer or just simple Route 53 load balancing using weighted AAA record.

  • shatrughanjha March 23, 2018

    I was unable to figure out the fix, so i have created my own load balancer using haproxy.

  • PasqualePuzio July 20, 2018

    Hi, any official answer on this subject?
    We plan to handle a lot of traffic so it's important to know in advance if there's a chance we're gonna hit any hard limit or not.

Log In to Comment
4 Answers
jtittle1 April 23, 2017

@yenthanh132

As far as I'm aware there's not a concrete limit on requests for the Load Balancer service.

That being said, a lot more goes in to handling numerous requests per second that just the LB. Since the LB is simply proxying the request to a server, or group of servers, and isn't actually handling the physical request for more than a split second, the limitation is most likely elsewhere (i.e. the sever(s) the request is being proxied to).

It's hard to really say either way without knowing more about the setup, configuration, and what we're working with.

Reply
timothyleland June 23, 2017

@yenthanh132 I seem to be having the same issue. After so many request the load balancer fails. Haven't been able to get an answer from DO about this. Had to switch my application to use cloudflare which hasn't had any issues but cost a lot more.

Reply
shatrughanjha January 2, 2018

I am having the same issue. I Just terminated the ssl on the load balancer and their tls handhsake took almost 16 sec with only 200 per second.

Reply
collimarco May 12, 2018

Yes, it's true. We have the same problem. Our backend can handle 70k client / s (7k * 10 servers), while the load balancer limits the throughput to only 1,000 client / s.

Reply
Have another answer? Share your knowledge.

Related Questions