How to support internal traffic with proxy protocol enabled on a Kubernetes loadbalancer
Enabling proxy protocol on a Kubernetes ingress load balancer only works with requests that come from outside the cluster. Requests from inside the cluster break.
This seems to be because requests from inside the cluster don’t get routed through the load balancer, so don’t get proxy protocol headers applied. The ingress controller then seems to fail to parse the request correctly, as it is expecting proxy protocol headers.
When proxy protocol is disabled, the ingress controller sees external traffic coming from a droplet subnet IP (presumably the loadbalancer IP), and internal traffic coming directly from a cluster pod subnet IP (interestingly, this isn’t the actual pod IP).
Note that this also means other DigitalOcean load balancer features don’t work for internal traffic, e.g. HTTPS redirection.
I haven’t been able to determine if this is due to the DigitalOcean LoadBalancer service implementation, or a feature of Kubernetes itself.
Either way, it means that I can’t use proxy protocol, as I have services that must be accessible inside and out of the cluster on the same hostname. It’d be great if there was some way to disable this behaviour, or some kind of workaround.
LoadBalancer service config for reference:
apiVersion: v1 kind: Service metadata: name: kong-proxy namespace: kong annotations: service.beta.kubernetes.io/do-loadbalancer-protocol: "http" service.beta.kubernetes.io/do-loadbalancer-algorithm: "round_robin" service.beta.kubernetes.io/do-loadbalancer-tls-ports: "443" service.beta.kubernetes.io/do-loadbalancer-tls-passthrough: "true" service.beta.kubernetes.io/do-loadbalancer-redirect-http-to-https: "true" service.beta.kubernetes.io/do-loadbalancer-enable-proxy-protocol: "true" spec: type: LoadBalancer selector: app: ingress-kong ports: - name: proxy port: 80 protocol: TCP targetPort: proxy - name: proxy-ssl port: 443 protocol: TCP targetPort: proxy-ssl externalTrafficPolicy: Local