Question
How to expose tcp port of kubernetes-nginx-ingress in DigitalOcean managed Kubernetes
- Posted October 8, 2020
- DigitalOcean Managed KubernetesNginx
I am following the below the guide
https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/
I also have deployed the nginx-ingress in kube-system namespace . These are the yamls files applied
apiVersion: v1
kind: ConfigMap
metadata:
name: tcp-nginx-rabbitmq
namespace: kube-system
data:
15672: "default/rabbitmq:15672"
apiVersion: v1
kind: Service
metadata:
annotations:
kompose.cmd: kompose convert --controller deployment -v -o DO-Kubernetes
kompose.version: 1.21.0 (992df58d8)
creationTimestamp: null
labels:
io.kompose.service: rabbitmq
name: rabbitmq
spec:
ports:
- name: "15672"
port: 15672
targetPort: 15672
- name: "5672"
port: 5672
targetPort: 5672
selector:
io.kompose.service: rabbitmq
status:
loadBalancer: {}
apiVersion: v1
kind: Service
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","kind":"Service","metadata":{"annotations":{"service.beta.kubernetes.io/do-loadbalancer-enable-proxy-protoco-nginx","app.kubernetes.io/managed-by":"Helm","app.kubernetes.io/name":"ingress-nginx","app.kubernetes.io/version":"0.35.0","helm.sh/externalTrafficPolicy":"Local","ports":[{"name":"http","port":80,"protocol":"TCP","targetPort":"http"},{"name":"https","port":443,"prrnetes.io/instance":"ingress-nginx","app.kubernetes.io/name":"ingress-nginx"},"type":"LoadBalancer"}}
kubernetes.digitalocean.com/load-balancer-id: <id>
service.beta.kubernetes.io/do-loadbalancer-enable-proxy-protocol: "true"
creationTimestamp: "2020-09-26T04:51:57Z"
finalizers:
- service.kubernetes.io/load-balancer-cleanup
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/version: 0.35.0
helm.sh/chart: ingress-nginx-2.13.0
name: ingress-nginx-controller
namespace: kube-system
resourceVersion: "11774380"
selfLink: /api/v1/namespaces/kube-system/services/ingress-nginx-controller
uid: <uid>
spec:
clusterIP: 10.245.75.126
externalTrafficPolicy: Local
healthCheckNodePort: 30093
ports:
- name: http
nodePort: 30583
port: 80
protocol: TCP
targetPort: http
- name: https
nodePort: 30628
port: 443
protocol: TCP
targetPort: https
- name: rabbitmq-tcp-15672
nodePort: 31555
port: 15672
protocol: TCP
targetPort: 15672
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
sessionAffinity: None
type: LoadBalancer
status:
loadBalancer:
ingress:
- ip: <ip>
The service is up and I can see the starting logs of rabbitmq ….(no hits) . The tcp port is also open from the loadbalancer confirmed with nc -zv. Firefox is also giving me an error
Secure Connection failed
ip:15672 PR_END_OF_FILE_ERROR
These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.
@pistle2020 How to expose multiple services using this tcp.
let say
apiVersion: v1 kind: ConfigMap metadata: name: tcp-services namespace: ingress-nginx-private data: 5432: “demo/postgres:5432”
this will expose “postgres” service. if we have 10 services which need to expose same like about how can we achieve this?
Ingress does not support TCP or UDP services. For this reason this Ingress controller uses the flags --tcp-services-configmap and --udp-services-configmap to point to an existing config map where the key is the external port to use and the value indicates the service to expose using the format: <namespace/service name>:<service port>:[PROXY]:[PROXY]
It is also possible to use a number or the name of the port. The two last fields are optional. Adding PROXY in either or both of the two last fields we can use Proxy Protocol decoding (listen) and/or encoding (proxy_pass) in a TCP service
The next example shows how to expose the service example-go running in the namespace default in the port 8080 using the port 9000
apiVersion: v1 kind: ConfigMap metadata: name: tcp-services namespace: ingress-nginx data: 9000: “default/example-go:8080” Since 1.9.13 NGINX provides UDP Load Balancing. The next example shows how to expose the service kube-dns running in the namespace kube-system in the port 53 using the port 53
apiVersion: v1 kind: ConfigMap metadata: name: udp-services namespace: ingress-nginx data: 53: “kube-system/kube-dns:53” If TCP/UDP proxy support is used, then those ports need to be exposed in the Service defined for the Ingress.
apiVersion: v1 kind: Service metadata: name: ingress-nginx namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx spec: type: LoadBalancer ports: - name: http port: 80 targetPort: 80 protocol: TCP - name: https port: 443 targetPort: 443 protocol: TCP - name: proxied-tcp-9000 port: 9000 targetPort: 9000 protocol: TCP selector: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx
@hcgaron This config works for me:
Word
PROXYis crucial in this line:@kennethafreelancer so am I . But it’s fun .
@kennethafreelancer
i am back again. Actually i got the https part working as well. in your case do a
and find the below lines
if u dont see --tcp-services-configmap add it like above.
cat nginx config
if u see some results then nginx knows about openvpn.
also discard my changes for removing port in configmap.
check nginx logs
i removed the port in configmap and nginx logs said invalid entry.
Following the steps in this thread I got the tcp working, but it broke my http. Specifically setting the proxy protocol to false breaks my http.
On the other hand, setting it to “true” breaks tcp forwarding. Any advice?
@favas Thanks man, I am new to kubernetes and I almost give up for implement this untill I saw your last message.
@favas Brother thanks ALOT!!! I can confirm this solution worked for this version of DO deployment of ingress-nginx https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.40.2/deploy/static/provider/do/deploy.yaml, so to expose the tcp service and http, https app using the same loadbalancer is completely possible
@favas Thanks for reply, still no luck lol
@kennethafreelancer I see that proxy_protocol annotation is enabled in the service yaml of ingress-nginx. Can you change that to false .
So it’s two changes remove port 35088 and the proxy_protocol to false
@favas tried, still have the same issue, the port is opened but not routed to openvpn. I tried expose openvpn with loadBalance will work, the reason I want to use nginx ingress is I want to keep only one load balancer.
@kennethafreelancer
can you try removing 35088 .
Openvpn service yaml says the targetport is 443 but I am assuming 35088 in the configmap is the issue.