Question

Why is Cluster Autoscaler not scaling down nodes?

I have a simple K8S cluster set up with the autoscaler enabled and one site set up with an HPA. When I run a load test on the site I can see more and more pods being provisioned and the cluster slowly scaling up nodes - in my test it scaled up to a total of 8 nodes.

The problem is that even though the HPA has brought the pods back down to from 20 to 2, it has been several hours and there is no sign of the autoscaler scaling down the cluster again. The only pods running on the new nodes are kube-system daemon sets (do-node-agent, csi-do-node, cpc-bridge-proxy, konnectivity-agent, cilium, kube-proxy)

Can someone point me in the right direction? I have been trying to find an answer but have gotten nowhere useful. I thought that daemon sets were ignored by the autoscaler in scale down decisions. Apologies in advance if this is a simple problem to resolve - I am new to K8S.


Submit an answer


This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

Sign In or Sign Up to Answer

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

I have the same issue. The autoscaler does not scale below 2, even though one of the nodes is completely empty except for pods in the kube-system namespace.

Bobby Iliev
Site Moderator
Site Moderator badge
December 15, 2022

Hi there,

This is quite interesting, usually it should take only a few minutes.

If you were to run the following command:

kubectl get configmap cluster-autoscaler-status -o yaml -n kube-system

Do you see any useful information? Feel free to share the output here after removing any sensitive details.

Best,

Bobby

Try DigitalOcean for free

Click below to sign up and get $200 of credit to try our products over 60 days!

Sign up

card icon
Get our biweekly newsletter

Sign up for Infrastructure as a Newsletter.

Sign up
card icon
Hollie's Hub for Good

Working on improving health and education, reducing inequality, and spurring economic growth? We’d like to help.

Learn more
card icon
Become a contributor

You get paid; we donate to tech nonprofits.

Learn more
Welcome to the developer cloud

DigitalOcean makes it simple to launch in the cloud and scale up as you grow – whether you’re running one virtual machine or ten thousand.

Learn more ->
DigitalOcean Cloud Control Panel
Get started for free

Enter your email to get $200 in credit for your first 60 days with DigitalOcean.

New accounts only. By submitting your email you agree to our Privacy Policy.

© 2023 DigitalOcean, LLC.