I have this scenario… 2 droplets on the same region, connected to Load Balancer…
How is that possible that when sending a request via PHP curl the API end point gets the droplet IP address? I want to split the amount of requests to the API I do per IP, so I don’t have issue of getting blocked… Is that possible?
Thanks in advance.
Greetings!
If I understand correctly, you have an outbound job on your droplet which reaches out to an external API. You want to balance that task so that the requests do not all come from the same IP. If I am understanding this correctly, there may not be one correct answer from a generic perspective that can be guaranteed to fit your situation.
These are a couple of thoughts:
If it were a cron job, for example, you could just do the cron job half as often from two servers.
If you created multiple proxies for the API requests you could then put those behind an haproxy instance. So like let’s say the endpoint is api.externalservice.com, I could make:
- haproxy (apiproxy.myservice.com)
- - proxy1 (proxy1.myservice.com) -> api.externalservice.com
- - proxy2 (proxy2.myservice.com) -> api.externalservice.com
Then I could have haproxy balance all outbound traffic through the two servers under it. Setting this up isn’t something I could directly advise on as I’m unfamiliar with your specific situation, but in theory a stack like this could function quite well. You can use haproxy for just about anything that requires load balancing. You can see me doing it with SMTP here: https://d.pr/i/4Lfbtq
Jarland
Hey Jarland,
Yes, this is exactly what I want… I want to balance the number of requests so not all requests comes from the same outbound IP…
I’m just trying to understand one thing about the load balancer… If I do a PHP curl request to the external API, does this API is gonna get the load balancer public IP or the droplet public API? Because if it gets the load balancer IP, it doesn’t matter from which droplet I make the request, the outbound IP is gonna be the same, right?
Am I correct or that is not how load balancing works?
Thanks for your reply.
Good thoughts. Let me see if I can communicate mine effectively (it’s a 50/50 shot some days, I get lost in my own head). So I made a diagram of a basic load balanced setup:
From here you see the LB doesn’t really play a role in outbound traffic initiated from within Web1 or Web2 servers. Any traffic leaving Web1 or Web2 leaves that server in the same way that it would if it were the only server in the stack. The outbound traffic from your droplet won’t go back out to the LB unless it’s requested through the LB.
So theoretically if I could initiate the API request by sending an HTTP request through the LB, maybe by calling a URL like “http://mywebsite.tld/executeapicall” then that would balance out between Web1 and Web2. But if the API call is simply executed on the droplet without being called to execute externally, it merely leaves that droplet and cares nothing about the LB that sits above it in the stack.
I hope that makes sense, I can easily get lost in this topic :)
To clarify further, at no point should an external API server see your load balancer with us as being the one making the requests, they should only see the droplet IP(s).
Though you could build a load balancer that they would see, in theory, just not the way ours are designed.
Oh okay, now I understood. I thought that the external API would see the load balancer IP instead of the droplet API.
So, if that is not true, my questions doesn’t make a lot of sense, hahaha…
Thank you very much for your quick replies!