I can no longer connect to external URLs after cluster upgrade

January 14, 2020 153 views

I upgraded my k8 cluster a few days ago and since then I have not been able to connect to external URLs.

I am getting a ‘connect timed out’ error when trying to load an external site and locally this same code is running properly.

Do I need to do something special to allow my pod to communicate externally? I messed around with some network policy stuff but that didn’t seem to help.

Could it be related to this change/fix?
1.16.2-do.2 (2020-01-10) :: Kubernetes Changelog

1 comment
  • Do you see all the pods in your kubesystem running healthy? I would namely look for coredns, kube-proxy, and cilium. If not I would ensure they have enough resources to run reliably and try kicking the pods again. Is the connection issue happening from all nodes?

2 Answers

kube-system all look healthy. This is happening in a cronjob and it has failed on each run/pod consistently since the cluster upgrade on Monday evening. Resources utilization on the cluster also looks good.

Ok to add more info this is only happening when trying to connect to sites that are running on the same cluster.

I run a test pod like this

kubectl run -it --rm --restart=Never alpine --image=alpine sh

And wget google.com works while wget brownbear.tech hangs.

Locally on my machine wget brownbear.tech works as expected along with following the HSTS policy to redirect to https.

It seems like the issue is a networking issue where it cannot properly route a URL which is also hosted on the cluster. For one of my sites hosted on a different cluster I do not see the issue making the connection.

Have another answer? Share your knowledge.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!