Scale up as you grow — whether you're running one virtual machine or ten thousand.

From GPU-powered inference and Kubernetes to managed databases and storage, get everything you need to build, scale, and deploy intelligent applications.

We have the following setup:
load-balancer/Ingress -> varnish -> service -> backends
What we see is that requests from varnish to the service (service as in “Kubernetes Service”) get routed to the same backend POD over and over again (maybe 99% of the time, that is less than 1 out of 100 requests get routed to a different backend POD).
In theory the kube-proxy provided by DO should chose a backend to route the IP to randomly, since DO are using the “iptables” variant of kube-proxy. However that’s not the case at all. The request nearly always gets routed to the same backend PODs.
The result is, that that one backend POD gets overwhelmed with requests and we need to do scaling inside the POD as opposed to using the HorizontalAutoscaler which would be best practice and which will automatically launch new PODs depending on load. However those new PODs will not get routed to by the service proxy (kube-proxy as I assume).
I am out of my depths. Since I have no influence over the configuration of kube-proxy, it being provided by DO, I have no influence on how the requests going to the service IP address get distributed by kube-proxy.
What can I do? Pointers? *t
PS: I can provide configs, but out setup being quite large, I don’t know which would be relevant here. I do not want to post all the config.
jazzybruno
jazzybruno
9c24208528c843f2bcdb7d54782c69
Sean DesPortes
Robin van der Knaap
Robin van der Knaap
Attila Kovacs
Tomy Hsieh
Tuschl
Alexander Freyr
marcelhaldimann
Emiliano Maccaferri