Question

Why is Cluster Autoscaler not scaling down nodes?

I have a simple K8S cluster set up with the autoscaler enabled and one site set up with an HPA. When I run a load test on the site I can see more and more pods being provisioned and the cluster slowly scaling up nodes - in my test it scaled up to a total of 8 nodes.

The problem is that even though the HPA has brought the pods back down to from 20 to 2, it has been several hours and there is no sign of the autoscaler scaling down the cluster again. The only pods running on the new nodes are kube-system daemon sets (do-node-agent, csi-do-node, cpc-bridge-proxy, konnectivity-agent, cilium, kube-proxy)

Can someone point me in the right direction? I have been trying to find an answer but have gotten nowhere useful. I thought that daemon sets were ignored by the autoscaler in scale down decisions. Apologies in advance if this is a simple problem to resolve - I am new to K8S.

Submit an answer


This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

Sign In or Sign Up to Answer

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

Bobby Iliev
Site Moderator
Site Moderator badge
• December 15, 2022

Hi there,

This is quite interesting, usually it should take only a few minutes.

If you were to run the following command:

kubectl get configmap cluster-autoscaler-status -o yaml -n kube-system

Do you see any useful information? Feel free to share the output here after removing any sensitive details.

Best,

Bobby

Try DigitalOcean for free

Click below to sign up and get $200 of credit to try our products over 60 days!

Sign up