How to support internal traffic with proxy protocol enabled on a Kubernetes loadbalancer

Enabling proxy protocol on a Kubernetes ingress load balancer only works with requests that come from outside the cluster. Requests from inside the cluster break.

This seems to be because requests from inside the cluster don’t get routed through the load balancer, so don’t get proxy protocol headers applied. The ingress controller then seems to fail to parse the request correctly, as it is expecting proxy protocol headers.

When proxy protocol is disabled, the ingress controller sees external traffic coming from a droplet subnet IP (presumably the loadbalancer IP), and internal traffic coming directly from a cluster pod subnet IP (interestingly, this isn’t the actual pod IP).

Note that this also means other DigitalOcean load balancer features don’t work for internal traffic, e.g. HTTPS redirection.

I haven’t been able to determine if this is due to the DigitalOcean LoadBalancer service implementation, or a feature of Kubernetes itself.

Either way, it means that I can’t use proxy protocol, as I have services that must be accessible inside and out of the cluster on the same hostname. It’d be great if there was some way to disable this behaviour, or some kind of workaround.

LoadBalancer service config for reference:

apiVersion: v1
kind: Service
metadata:
  name: kong-proxy
  namespace: kong
  annotations:
    service.beta.kubernetes.io/do-loadbalancer-protocol: "http"
    service.beta.kubernetes.io/do-loadbalancer-algorithm: "round_robin"
    service.beta.kubernetes.io/do-loadbalancer-tls-ports: "443"
    service.beta.kubernetes.io/do-loadbalancer-tls-passthrough: "true"
    service.beta.kubernetes.io/do-loadbalancer-redirect-http-to-https: "true"
    service.beta.kubernetes.io/do-loadbalancer-enable-proxy-protocol: "true"
spec:
  type: LoadBalancer
  selector:
    app: ingress-kong
  ports:
  - name: proxy
    port: 80
    protocol: TCP
    targetPort: proxy
  - name: proxy-ssl
    port: 443
    protocol: TCP
    targetPort: proxy-ssl
  externalTrafficPolicy: Local