Can't get load balancer in a healthy state

October 28, 2018 2.1k views
Load Balancing Kubernetes
vegemike
By:
vegemike

Howdy! I’m trying out the k8s beta and it’s been great so far. I have two clusters, both with load balancers sitting in front. One of the load balancers is working great and has had no issue. The second hasn’t been in a healthy state since I created it. The droplets are hosting a third party app that doesn’t respond to /, but it does have another healthcheck URL.

The solution it seems was to put nginx in front of the app, which I’ve done and it responds with a 200 on /, but the load balancer still thinks the droplet is unhealthy. I’ve tried various configurations for TCP and HTTP checks on various ports.

Using kubectl proxy, I’ve been able to get a 200 response from the / endpoint, so I’m not really sure what else I need to do. Any help would be great. For reference, here’s my nginx and k8s service config: https://gist.github.com/mike-engel/a1c11cd6d0edcdab6fd912619b33d972

Thanks!

Sign In to Comment
3 Answers
vegemike October 31, 2018

I think I have this solved. The solution, I think, involved two parts. The first is that I added a selector to my service config that targeted the pod via a label. The second was that my nginx config was setting the server_name to localhost rather than 0.0.0.0. I think 0.0.0.0 is required for k8s deployments.

Report
unixynet October 29, 2018

The LB does a HEAD. Make sure your proxy returns the proper HTTP code for that or can translate it to a GET.

Report
vegemike October 30, 2018

@unixynet Thansk for the reply! With my nginx config, however, running a HEAD / successfully returns a 200. Here’s the output from cURL.

curl -isb -X HEAD http://127.0.0.1:8001/api/v1/namespaces/default/pods/pod-id/proxy/

HTTP/1.1 200 OK
Audit-Id: 475e996e-99fd-4303-9d0f-01ea240c68ef
Content-Length: 2
Content-Type: text/plain; charset=utf-8
Date: Tue, 30 Oct 2018 01:40:00 GMT
Server: nginx/1.14.0

OK
Report
Have another answer? Share your knowledge.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

Related