Question

How to configure DO K8S Managed Load Balancers properly

Posted May 5, 2021 157 views
Load BalancingKubernetesDigitalOcean Managed KubernetesDigitalOcean Managed Load Balancers

I have installed nginx-ingress from here: https://www.digitalocean.com/community/tutorials/how-to-set-up-an-nginx-ingress-with-cert-manager-on-digitalocean-kubernetes.

I have provisioned and configured my Load Balancer on my K8S cluster manually through the Control Panel.

I have created an appropriate Ingress object in my project. All this works. For a while.

As it turns out though, manual configurations on the Load Balancer don’t last long. The DO Cloud Control Manager (CCM) regularly updates the Load Balancer configuration from what it thinks would be best. That overrides my settings (specifically the port numbers) and does not work.

The way to properly configure the Load Balancer is apparently by annotating “the” LoadBalancer Service object. Seen here: https://docs.digitalocean.com/products/kubernetes/how-to/configure-load-balancers/

The problem is, that LoadBalancer service object is created by the nginx-ingress script above (named “nginx-ingress-ingress-nginx-controller”). I don’t own that, it is not managed by my k8s yaml scripts, it’s not even in my namespace.

What is the appropriate process here? Have I missed something? Do I need to own the LoadBalancer service? Should I just patch/overwrite it until it works? Do I leave the load-balancer-id annotation? Do I leave the helm annotations too? (I don’t use Helm) What happens if that script gets updated?

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
Submit an Answer
1 answer

Hello @brautigamrobertgyula

All loadbalancers provisioned by kubernetes are managed by the Cloud Controller Manager(CCM) running on the master. Any manual modifications to the LB through the cloud panel will be overwritten by the CCM during its reconciliation process to ensure the LB is configured properly with the state defined by the kubernetes service.

In order to configure your LB you need to use kubernetes service annotations. For information on how to apply annotations, you can use the command:

kubectl annotate --help

Docs for DO service annotations can be found here: https://www.digitalocean.com/docs/kubernetes/how-to/configure-load-balancers/

Service annotations can also be found on our public github repository:

https://github.com/digitalocean/digitalocean-cloud-controller-manager/blob/master/docs/controllers/services/annotations.md

I hope this helps!

Best Regards,
Purnima Kumari

  • Thanks for the information. As it turns out annotations don’t solve the problem, because the annotations don’t seem to control the target of the routing.

    So in my case because of the SSL termination, HTTPS should route to HTTP. It doesn’t for some reason.

    I ended up modifying the LoadBalancer service itself (which doesn’t feel right). So this:

    ports:
      - name: http
        nodePort: <port1>
        port: 80
        protocol: TCP 
        targetPort: http
      - name: https
        nodePort: <port2>
        port: 443 
        protocol: TCP 
        targetPort: https
    

    Became this:

    ports:
      - name: http
        nodePort: <port1>
        port: 80
        protocol: TCP 
        targetPort: http
      - name: https
        nodePort: <port2>
        port: 443 
        protocol: TCP 
        targetPort: http <-- This instead of "https"
    

    I have to admit, I have no clue what consequences this has. But it seems to work.

    Note: It only works from the “outside”. From the “inside” it is broken now, because now the https port directly maps to the http port, so there is no SSL/TLS at all (I assume it doesn’t really go through the loadbalancer from the inside).

    I don’t consider this a solution, more of a workaround. If anyone knows a better way, that would be awesome.