Question

How to automatically add a domain to a LoadBalancer Service on Kubernetes using Terraform?

Posted December 27, 2020 1.9k views
DNSTerraformDigitalOcean Managed Kubernetes

GOAL:
I’m using terraform to quickly and easily setup a Kubernetes cluster. I have a LoadBalancer service that should expose my blog-backend service with a custom domain (I have already added DNS NS entries in my Cloudflare that redirects base domain to DigitalOcean’s servers).

resource "kubernetes_service" "blog-backend-lb" {
  metadata {
    name = "blog-backend-lb"
    labels = {
      app = "blog-backend"
    }
    annotations = {
      "service.beta.kubernetes.io/do-loadbalancer-name" = "blog.mydomain.com"
      "service.beta.kubernetes.io/do-loadbalancer-hostname" = "blog.mydomain.com"
    }
  }
  spec {
    type = "LoadBalancer"

    selector = {
      app = "blog-backend"
    }

    port {
      port = 8080
      target_port = 8080
    }
  }
}

PROBLEM:
The service is being create properly and I can see Kubernetes picked up a domain.

041

However, I cannot access it unless I manually create a domain entry in the DigitalOcean dashboard: Networking -> Domains -> Add Domain -> Create Record for my subdomain redirecting to a LoadBalancer.

This is a bit cumbersome, because I want to clean everything up using terraform without touching anything by hand.

QUESTION:
How can I automate it using a Terraform?

I thought about using a domain resource, but I don’t know where to get the ip address from.

resource "digitalocean_domain" "blog-backend-domain" {
  name = "blog.mydomain.com"
  ip_address = "<WHERE TO GET THE IP FROM?>"
}

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
Submit an Answer
2 answers

Ok, problem solved. When I used LoadBalancer as a service I didn’t have access to its IP from within terraform. However, when using ingress-nginx helm chart I have the IP. Yay!

So the solution is:

# 1. Download ingress-nginx chart
resource "helm_release" "ingress-nginx" {
  repository = "https://kubernetes.github.io/ingress-nginx"
  chart = "ingress-nginx"
  name = "ingress-nginx"
}

# 2. Add a Service with `type = "ClusterIP"`, which would expose the app to the Kubernetes cluster
resource "kubernetes_service" "blog-backend" {
  metadata {
    name = "blog-backend-service"
    labels = {
      app = "blog-backend"
    }
  }
  spec {
    type = "ClusterIP"

    selector = {
      app = "blog-backend"
    }

    port {
      port = 8080
      target_port = 8080
    }
  }
}

# 3. Create an ingress controller, which maps a domain to a service above
resource "kubernetes_ingress" "blog-backend-ingress" {
  metadata {
    name = "blog-backend-ingress"
    annotations = {
      "kubernetes.io/ingress.class" = "nginx"
    }
  }
  spec {
    rule {
      host = "blog.mydomain.com"
      http {
        path {
          backend {
            service_name = "blog-backend-service"
            service_port = 8080
          }
          path = "/"
        }
      }
    }
  }
}

# 4. Create a domain
resource "digitalocean_domain" "blog-backend-domain" {
  name = "blog.mydomain.com"
}

# 5. Create a domain record, whose IP (`value`) we get from the ingress from step 3.
resource "digitalocean_record" "backend-api" {
  domain = digitalocean_domain.blog-backend-domain.name
  name = "@"
  type = "A"
  value = kubernetes_ingress.blog-backend-ingress.load_balancer_ingress[0].ip
}
  • The script above will work only when the Load Balancer is already created. However, if we set up the cluster from scratch then it takes some time to assign public ip to the Load Balancer. To fix it we need to add wait_for_load_balancer = true to our backend ingress.

    # 3. Create an ingress controller, which maps a domain to a service above
    resource "kubernetes_ingress" "blog-backend-ingress" {
      ##### ADD THIS THING #####
      wait_for_load_balancer = true
      ##########################
      metadata {
        name = "blog-backend-ingress"
        annotations = {
          "kubernetes.io/ingress.class" = "nginx"
        }
      }
      spec {
        rule {
          host = "blog.mydomain.com"
          http {
            path {
              backend {
                service_name = "blog-backend-service"
                service_port = 8080
              }
              path = "/"
            }
          }
        }
      }
    }
    
    

The latest Terraform version has modified the way we get IP from ingress

output "load_balancer_ip" {
  value = kubernetes_ingress.example.status.0.load_balancer.0.ingress.0.ip
}

So use this block


resource "digitalocean_record" "backend-api" {
  domain = digitalocean_domain.blog-backend-domain.name
  name = "@"
  type = "A"
  value = kubernetes_ingress.blog-backend-ingress.status.0.load_balancer.0.ingress.0.ip
 }