Question

How can I get Socket.io to work on Kubernetes when using nginx-ingress and cert-manager?

I have a deployment running inside my k8s cluster that uses Socket.io, but external WebSocket traffic doesn’t seem to be getting to it.

I’m using nginx ingress controller with cert-manager, which works fine for normal HTTPS traffic. With this setup, SSL termination is with nginx and the certificates live in the cluster.

I’ve seen in the docs and elsewhere that I need to switch the load balancer protocol to HTTP instead of TCP to get WebSockets to work. However, doing that breaks SSL for me. This is because the DigitalOcean load balancer has two options:

  1. SSL Termination at the load balancer - this would require managing my domain and SSL certs through DigitalOcean and abandoning cert-manager which I don’t want to do
  2. SSL Passthrough - this passes the encrypted data straight to the droplet, which is also not what I want to do (I think??)

Is there a way to continue to use nginx ingress controller and cert manager with WebSockets?

Show comments

Submit an answer

This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

Sign In or Sign Up to Answer

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

I got this working in the end, terminating the SSL at nginx (passthrough on the load balancer) and allowing it to reverse-proxy the data to the apps with sticky sessions. For anyone else stuck on this, here are my manifests - hope they help:

nginx-ingress-loadbalancer-service.yaml

kind: Service
apiVersion: v1
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  annotations: 
    # See https://github.com/digitalocean/digitalocean-cloud-controller-manager/blob/master/docs/controllers/services/examples/README.md#accessing-pods-over-a-managed-load-balancer-from-inside-the-cluster
    service.beta.kubernetes.io/do-loadbalancer-name: "api.example.com"
    service.beta.kubernetes.io/do-loadbalancer-hostname: "api.example.com"
    service.beta.kubernetes.io/do-loadbalancer-protocol: "http"
    service.beta.kubernetes.io/do-loadbalancer-tls-passthrough: "true"
    service.beta.kubernetes.io/do-loadbalancer-sticky-sessions-type: "cookies"
    service.beta.kubernetes.io/do-loadbalancer-sticky-sessions-cookie-name: "jammydodger"
    service.beta.kubernetes.io/do-loadbalancer-sticky-sessions-cookie-ttl: "60"
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  externalTrafficPolicy: Local
  type: LoadBalancer
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: http
    - name: https
      port: 443
      protocol: TCP
      targetPort: https

---

ingress.yaml

# Define Ingress for the service
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: testing-ingress
  annotations:
    kubernetes.io/ingress.class: "nginx"
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
    nginx.ingress.kubernetes.io/rewrite-target: /$2 # Lets us rewrite example.com/something/hello to example.com/hello
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
    nginx.ingress.kubernetes.io/affinity: "cookie"
    nginx.ingress.kubernetes.io/session-cookie-name: "route"
    nginx.ingress.kubernetes.io/session-cookie-hash: "sha1"
    nginx.ingress.kubernetes.io/session-cookie-expires: "172800"
    nginx.ingress.kubernetes.io/session-cookie-max-age: "172800"

spec:
  tls:
  - hosts:
    - api.example.com
    - example.com
    secretName: testing-ingress-tls
  rules:
  - host: api.example.com
    http:
      paths:
      - path: /v0(/|$)(.*)
        backend:
          serviceName: platform-app-service
          servicePort: 80
  - host: example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: frontend-app-service
          servicePort: 80

Did you ever get a resolution to this, I am stuck with exactly the same issue.