Kubernetes. Unable to connect to the server

Posted March 4, 2020 22.5k views

This is the third time when Kubernetes starts to be unavailable for “kubectl”. Sometimes I cannot connect to the cluster using Kubernetes CLI and get the following errors:

Unable to connect to the server: dial tcp x.x.x.x:443: i/o timeout


Unable to connect to the server: net/http: TLS handshake timeout

The indicator near the k8s logo lights yellow whereas it’s green when all Okey. When I try to add extra nodes it stucks in “loading” state without any changes. We rely on your servers and clusters, but it lets us down every week. On your servers, we host production environments of projects and monitoring infrastructure.
I cannot Google any information about this issue.
What do we do wrong? How to avoid these problems in the future?

  • Did anyone find a solution to this? I am having the exact issue with one of my clusters today.

  • I’m having this issue as well. Not with anything crazy or particularly API heavy. I can’t imagine what would be causing these kinds of problems. I’ve enabled auto-scaling on my pool and all its running is a proxy and monitoring.

  • Woke up to an alert from a service that I can’t access thanks to this issue. Kubectl just times out making a request, as does ping (!). No indication of any issue on DO’s status site or dashboard (not including the k8s-built dashboard, which, surprise!, can’t connect). It seems to be completely out of my hands.

    I haven’t touched any k8s config in months, this is completely out of the blue. Node metrics are happy, and thankfully things remain up, but to say I am concerned about the stability of my production kubernetes hosting would be an understatement.

  • Not sure if this is the same issue but when our cluster scales up or down due to auto-scaling our load balancer goes down and is unable to connect the pods for around 10s (health checks on do lb show the nodes are down). This causes incoming data streams to break and is a big problem for us.

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

Submit an Answer
17 answers

The same problem.
Cluster is not accessible for the last 2 hours.
Apps running on cluster are not accessible too.

Managed Kubernetes in DO is not production ready at all :(

As of today is still happening, when installing prometheus-operator chart it loses the connectivity to the cluster to the point where any kubectl command it’s not working.

Even a single kubectl apply -f for a single resource, triggered the connectivity loss to the cluster for more than 2 hours.

And adding nodes to the current pool, took more than 3 hours to complete.

I hope we get more transparency of what is happening with our clusters.

Hi there!

This can occur if you have any API heavy workloads deployed that put strain on the master node. If you want you can open a support ticket and I can dig in to see whats occurring on your cluster’s control plane.


John Kwiatkoski
Senior Developer Support Engineer - Kubernetes

Did you find a solution to this? I’ve had this issue twice today, and droplets disappearing from the load balancer as well. Popped in a ticket, and have had no reply.

Feels like the Digital Ocean’s Kubernetes solution is not quite production-ready.

Same problem. Tasks like “helm upgrade” for a simple Prometheus installation cause the DO k8s going yellow in DO panel and cannot get any answer to kubectl commands. There was not heavy use of k8s API.

It seems to me that DO k8s is not production ready yet.

This happened to me for the 4th or 5th time last night when upgrading the Prometheus Operator helm chart, which is set up according to the DO guide. No response to my ticket for the last 15 hours. This would be utterly unacceptable if I were in production, so I probably won’t be able to use DOKS for my production needs.

Last time I put in a ticket about this, they restarted the master and basically said I’d need to spend more money on nodes to get a better master instance (without saying how where the breakpoints on that are).

I’ve just experienced this issue for 4-th time and each time I’ve ended up creating a new cluster. I am personally using K8s for my own side-project and it’s still very frustrating even though it isn’t real production yet.

This time the issue started to kick in imminently after I’ve tried to install Kubernetes Monitoring Stack.
It never even fully installed, just created Prometheus Operator namespace and its secret.
It seems like the issue is somehow related to Prometheus Operator as many others already noticed.

So I’ve googled both of these terms together and found this blog post that I think actually reveals and solves the mystery. And although it intended for GKE it still might help newcomers to deal with this problem.

edited by MattIPv4

Same here.

Cluster is not accessible since 16hours now (sometimes it works, you have to try it about 50 times to have a single lucky connection).

support tickets are simply ignored or answered with “everything seems to work”.

Sorry, managed kubernetes in DO seems to be toy. and the support too.

The same issue, clusters change from ready to unready and no way to debug is available not even shell over the browser. Total invisibility on what is going on.

Completely agree. Managed Kubernetes in DO is not production ready at all :(
I have the same problem.
Fortunately I tested it in test mode.

Previous 1 2 Next