Report this

What is the reason for this report?

Deploying ingress-nginx on new cluster creates load balancer that fails health check

Posted on October 28, 2020

When I deploy ingress-nginx, a load balancer is created that points to two nodes. One healthy and one down.

Using this command: kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.40.2/deploy/static/provider/do/deploy.yaml

Following this guide: https://www.digitalocean.com/community/tutorials/how-to-set-up-an-nginx-ingress-with-cert-manager-on-digitalocean-kubernetes

Do I need to configure this further? Not sure where the problem is, or how one is successful and one fails. I’ve done this many times with the same results.

Steps to reproduce:

  1. Create a cluster (I did 2 nodes)
  2. run kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.40.2/deploy/static/provider/do/deploy.yaml
  3. Check new load balancer


This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

I had the same problem.

The loadbalancer health check port is checking the port of ingress-nginx service. Because ingress-nginx service is running only on one node all other nodes will be shown as down.

You need to change the healthcheck port. I changed mine to the port of node-exporter which is running on every node as a daemon-set.

I just ended up using the the nginx ingress from the marketplace instead and it worked fine

I was running into this issue, and from what it seems, the load balancer will run a health check at the NodePort on the ingress-nginx-controller at the http:80 destination, but for some reason that only seems to run/live on one node.

It turns out (for me) a HealthCheck NodePort is created, and works, but the load balancer will need to be manually updated run health checks on that. Example output:

➜ ~ kubectl -n ingress-nginx describe svc ingress-nginx-controller
Name:                     ingress-nginx-controller
Namespace:                ingress-nginx
# [blah blah blah..]
Port:                     http  80/TCP
TargetPort:               http/TCP
NodePort:                 http  31869/TCP # <= LB Health check targets this
Endpoints:                10.244.3.183:80
Port:                     https  443/TCP
TargetPort:               https/TCP
NodePort:                 https  31015/TCP
Endpoints:                10.244.3.183:443
Session Affinity:         None
External Traffic Policy:  Local
HealthCheck NodePort:     31387 # <= LB Health check should target that

I also did not have this issue with the marketplace/one-click app version on the nginx ingress controller.

The developer cloud

Scale up as you grow — whether you're running one virtual machine or ten thousand.

Get started for free

Sign up and get $200 in credit for your first 60 days with DigitalOcean.*

*This promotional offer applies to new accounts only.