Cannot obtain letsencrypt certificate with ClusterIssuer

November 6, 2019 121 views
Kubernetes

Hello,
I’ve created brand new cluster to deploy some node.js services. I followed up on the following tutorial:
https://www.digitalocean.com/community/tutorials/how-to-set-up-an-nginx-ingress-on-digitalocean-kubernetes-using-helm

Unfortunately I have some issues with “Step 4 — Securing the Ingress Using Cert-Manager”.

What I did

I’ve manually applied CRD’s, Jetstack Helm repository and installed cert-manager as proposed in the article.

Then I defined:

apiVersion: certmanager.k8s.io/v1alpha1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    # The ACME server URL
    server: https://acme-v02.api.letsencrypt.org/directory
    # Email address used for ACME registration
    email: x.y@gmail.com
    # Name of a secret used to store the ACME account private key
    privateKeySecretRef:
      name: letsencrypt-prod
    # Enable the HTTP-01 challenge provider
    http01: {}

and created cluster issuer. Here are the results:

x@y-pc:~/x/y/z$ kubectl create -f production_issuer.yaml
clusterissuer.certmanager.k8s.io/letsencrypt-prod created
x@y-pc:~/x/y/z$ kubectl describe clusterissuers letsencrypt-prod
Name:         letsencrypt-prod
Namespace:    
Labels:       <none>
Annotations:  <none>
API Version:  certmanager.k8s.io/v1alpha1
Kind:         ClusterIssuer
Metadata:
  Creation Timestamp:  2019-11-06T21:54:43Z
  Generation:          1
  Resource Version:    2797607
  Self Link:           /apis/certmanager.k8s.io/v1alpha1/clusterissuers/letsencrypt-prod
  UID:                 d71fc683-a..3-4..f-92d5-ff........20
Spec:
  Acme:
    Email:  x.y@gmail.com
    http01:
    Private Key Secret Ref:
      Name:  letsencrypt-prod
    Server:  https://acme-v02.api.letsencrypt.org/directory
Events:      <none>

So far so good.
Then I defined:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: node-test-app
  annotations:
    kubernetes.io/ingress.class: nginx
    certmanager.k8s.io/cluster-issuer: letsencrypt-prod
spec:
  tls:
  - hosts:
    - node-test-app.x.y.z.pl
    secretName: letsencrypt-prod
  rules:
  - host: node-test-app.x.y.z.pl
    http:
      paths:
      - backend:
          serviceName: node-test-app
          servicePort: 80

and created (updated) my ingress with the results:

x@y-pc:~/x/y/z$ kubectl apply -f ingress.yaml --namespace=playground
ingress.extensions/node-test-app created

The problem

I cannot obtain the certificate:

x@y-pc:~/x/y/z$ kubectl describe certificate letsencrypt-prod
Error from server (NotFound): certificates.certmanager.k8s.io "letsencrypt-prod" not found
x@y-pc:~/x/y/z$ kubectl get ingress --namespace=playground
NAME            HOSTS                                ADDRESS         PORTS     AGE
node-test-app   node-test-app.x.y.z.pl   67.207.x.y   80, 443   2m44s

x@y-pc:~/x/y/z$ kubectl get ingress node-test-app -o yaml --namespace=playground
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    certmanager.k8s.io/cluster-issuer: letsencrypt-prod
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"certmanager.k8s.io/cluster-issuer":"letsencrypt-prod","kubernetes.io/ingress.class":"nginx"},"name":"node-test-app","namespace":"playground"},"spec":{"rules":[{"host":"node-test-app.x.y.z.pl","http":{"paths":[{"backend":{"serviceName":"node-test-app","servicePort":80}}]}}],"tls":[{"hosts":["node-test-app.x.y.z.pl"],"secretName":"letsencrypt-prod"}]}}
    kubernetes.io/ingress.class: nginx
  creationTimestamp: "2019-11-06T22:02:08Z"
  generation: 1
  name: node-test-app
  namespace: playground
  resourceVersion: "2798216"
  selfLink: /apis/extensions/v1beta1/namespaces/playground/ingresses/node-test-app
  uid: 5d33088a-2d10-x-y-z
spec:
  rules:
  - host: node-test-app.x.y.z.pl
    http:
      paths:
      - backend:
          serviceName: node-test-app
          servicePort: 80
  tls:
  - hosts:
    - node-test-app.x.y.z.pl
    secretName: letsencrypt-prod
status:
  loadBalancer:
    ingress:
    - ip: 67.207.x.y

x@y-pc:~/x/y/z$ kubectl describe  ingress node-test-app --namespace=playground
Name:             node-test-app
Namespace:        playground
Address:          67.207.x.y
Default backend:  default-http-backend:80 (<none>)
TLS:
  letsencrypt-prod terminates node-test-app.x.y.z.pl
Rules:
  Host                                Path  Backends
  ----                                ----  --------
  node-test-app.x.y.x.pl  
                                         node-test-app:80 (10.244.x.y:3000)
Annotations:
  certmanager.k8s.io/cluster-issuer:                 letsencrypt-prod
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"certmanager.k8s.io/cluster-issuer":"letsencrypt-prod","kubernetes.io/ingress.class":"nginx"},"name":"node-test-app","namespace":"playground"},"spec":{"rules":[{"host":"node-test-app.x.y.z.pl","http":{"paths":[{"backend":{"serviceName":"node-test-app","servicePort":80}}]}}],"tls":[{"hosts":["node-test-app.x.y.z.pl"],"secretName":"letsencrypt-prod"}]}}

  kubernetes.io/ingress.class:  nginx
Events:
  Type    Reason  Age    From                      Message
  ----    ------  ----   ----                      -------
  Normal  CREATE  5m26s  nginx-ingress-controller  Ingress playground/node-test-app
  Normal  UPDATE  4m34s  nginx-ingress-controller  Ingress playground/node-test-app

x@y-pc:~/x/y/z$ kubectl get certificates --namespace=playground
No resources found in playground namespace.

Can you help me tackle that issue? Where should I start with debugging?

1 Answer

Basically I was using outdated version of cert-manager. Article mentioned above should be updated and that link should be changed:

kubectl apply -f https://raw.githubusercontent.com/jetstack/cert-manager/release-0.8/deploy/manifests/00-crds.yaml

At the moment of writing that comment release 1.1 was available.

On the other hand I still have another problems with my certificate.
After I run

x@y-pc:~/x/y/z$ kubectl describe certificate letsencrypt-playground-node-test-app -n playground

I got the following results:

Name:         letsencrypt-playground-node-test-app
Namespace:    playground
Labels:       <none>
Annotations:  <none>
API Version:  cert-manager.io/v1alpha2
Kind:         Certificate
Metadata:
  Creation Timestamp:  2019-11-08T16:40:18Z
  Generation:          1
  Owner References:
    API Version:           extensions/v1beta1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Ingress
    Name:                  node-test-app
    UID:                   9fd3b1ea-9832-4262-9ab5-20dc87c301a5
  Resource Version:        3026499
  Self Link:               /apis/cert-manager.io/v1alpha2/namespaces/playground/certificates/letsencrypt-playground-node-test-app
  UID:                     1583b21b-9957-4a96-a7f6-b7d025420db7
Spec:
  Dns Names:
    node-test-app.playground.x.y.z
  Issuer Ref:
    Group:      cert-manager.io
    Kind:       Issuer
    Name:       letsencrypt-staging
  Secret Name:  letsencrypt-playground-node-test-app
Status:
  Conditions:
    Last Transition Time:  2019-11-08T16:40:44Z
    Message:               Certificate is up to date and has not expired
    Reason:                Ready
    Status:                True
    Type:                  Ready
  Not After:               2020-02-06T15:40:43Z
Events:
  Type    Reason        Age   From          Message
  ----    ------        ----  ----          -------
  Normal  GeneratedKey  19m   cert-manager  Generated a new private key
  Normal  Requested     19m   cert-manager  Created new CertificateRequest resource "letsencrypt-playground-node-test-app-4005185548"
  Normal  Issued        19m   cert-manager  Certificate issued successfully

Everything seems to be fine, but my page still does not have certificate and I still see the warning about missing certificate in the web browser.

Any help would be still appreciated!

PS. If anyone of you have any problems with cert-manager I strongly recommend to check logs of cert-manager.
kubectl get pods -n cert-manager
and then using results of that command:
`kubectl logs cert-manager-XXX -n cert-manager

Have another answer? Share your knowledge.