Related

Join the DigitalOcean Community Community

Join 1M+ other developers and:

  • Get help and share knowledge in Q&A
  • Subscribe to topics of interest
  • Get courses & tools that help you grow as a developer or small business owner
 Join Now
Will kubernetes have autoscaling by default? Question Question
Firewall blocks hostPort Question Question

Question

Kubernetes. Unable to connect to the server

Posted March 4, 2020 18k views
Kubernetes

This is the third time when Kubernetes starts to be unavailable for “kubectl”. Sometimes I cannot connect to the cluster using Kubernetes CLI and get the following errors:

Unable to connect to the server: dial tcp x.x.x.x:443: i/o timeout

Or 

Unable to connect to the server: net/http: TLS handshake timeout

The indicator near the k8s logo lights yellow whereas it’s green when all Okey. When I try to add extra nodes it stucks in “loading” state without any changes. We rely on your servers and clusters, but it lets us down every week. On your servers, we host production environments of projects and monitoring infrastructure.
I cannot Google any information about this issue.
What do we do wrong? How to avoid these problems in the future?

Add a comment
vitalyaxmit
By:
vitalyaxmit

Related

Join the DigitalOcean Community Community

Join 1M+ other developers and:

  • Get help and share knowledge in Q&A
  • Subscribe to topics of interest
  • Get courses & tools that help you grow as a developer or small business owner
 Join Now
Will kubernetes have autoscaling by default? Question Question
Firewall blocks hostPort Question Question
Add a comment
4 comments
  • maxbaun April 14, 2020
    Reply Report

    Did anyone find a solution to this? I am having the exact issue with one of my clusters today.

  • lorxraposa April 15, 2020
    Reply Report

    I’m having this issue as well. Not with anything crazy or particularly API heavy. I can’t imagine what would be causing these kinds of problems. I’ve enabled auto-scaling on my pool and all its running is a proxy and monitoring.

  • adambard November 7, 2020
    Reply Report

    Woke up to an alert from a service that I can’t access thanks to this issue. Kubectl just times out making a request, as does ping (!). No indication of any issue on DO’s status site or dashboard (not including the k8s-built dashboard, which, surprise!, can’t connect). It seems to be completely out of my hands.

    I haven’t touched any k8s config in months, this is completely out of the blue. Node metrics are happy, and thankfully things remain up, but to say I am concerned about the stability of my production kubernetes hosting would be an understatement.

  • noahschu April 29, 2021
    Reply Report

    Not sure if this is the same issue but when our cluster scales up or down due to auto-scaling our load balancer goes down and is unable to connect the pods for around 10s (health checks on do lb show the nodes are down). This causes incoming data streams to break and is a big problem for us.

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
Submit an Answer
17 answers
swhile April 18, 2020

The same problem.
Cluster is not accessible for the last 2 hours.
Apps running on cluster are not accessible too.

Managed Kubernetes in DO is not production ready at all :(

Reply Report
christiancuaresma July 10, 2020

As of today is still happening, when installing prometheus-operator chart it loses the connectivity to the cluster to the point where any kubectl command it’s not working.

Even a single kubectl apply -f for a single resource, triggered the connectivity loss to the cluster for more than 2 hours.

And adding nodes to the current pool, took more than 3 hours to complete.

I hope we get more transparency of what is happening with our clusters.

Reply Report
jkwiatkoski March 4, 2020

Hi there!

This can occur if you have any API heavy workloads deployed that put strain on the master node. If you want you can open a support ticket and I can dig in to see whats occurring on your cluster’s control plane.

Regards,

John Kwiatkoski
Senior Developer Support Engineer - Kubernetes

Reply Report
anopier April 8, 2020

Did you find a solution to this? I’ve had this issue twice today, and droplets disappearing from the load balancer as well. Popped in a ticket, and have had no reply.

Feels like the Digital Ocean’s Kubernetes solution is not quite production-ready.

Reply Report
michaelbenardis April 23, 2020

Same problem. Tasks like “helm upgrade” for a simple Prometheus installation cause the DO k8s going yellow in DO panel and cannot get any answer to kubectl commands. There was not heavy use of k8s API.

It seems to me that DO k8s is not production ready yet.

Reply Report
jyorke July 19, 2020

This happened to me for the 4th or 5th time last night when upgrading the Prometheus Operator helm chart, which is set up according to the DO guide. No response to my ticket for the last 15 hours. This would be utterly unacceptable if I were in production, so I probably won’t be able to use DOKS for my production needs.

Last time I put in a ticket about this, they restarted the master and basically said I’d need to spend more money on nodes to get a better master instance (without saying how where the breakpoints on that are).

Reply Report
timothyy October 20, 2020

I’ve just experienced this issue for 4-th time and each time I’ve ended up creating a new cluster. I am personally using K8s for my own side-project and it’s still very frustrating even though it isn’t real production yet.

This time the issue started to kick in imminently after I’ve tried to install Kubernetes Monitoring Stack.
It never even fully installed, just created Prometheus Operator namespace and its secret.
It seems like the issue is somehow related to Prometheus Operator as many others already noticed.

So I’ve googled both of these terms together and found this blog post that I think actually reveals and solves the mystery. And although it intended for GKE it still might help newcomers to deal with this problem.

edited by MattIPv4
Reply Report
usc April 21, 2020

Same here.

Cluster is not accessible since 16hours now (sometimes it works, you have to try it about 50 times to have a single lucky connection).

support tickets are simply ignored or answered with “everything seems to work”.

Sorry, managed kubernetes in DO seems to be toy. and the support too.

Reply Report
ndubidarius April 23, 2020

The same issue, clusters change from ready to unready and no way to debug is available not even shell over the browser. Total invisibility on what is going on.

Reply Report
arbusculaShark May 22, 2020

Completely agree. Managed Kubernetes in DO is not production ready at all :(
I have the same problem.
Fortunately I tested it in test mode.

Reply Report
Previous 1 2 Next
Load More Answers

Related

Looking for something else?

Ask a new question Search for more help