Question

Garbage Collection on Kubernetes Cluster Not Working

Posted October 4, 2019 755 views
Kubernetes DigitalOcean Managed Kubernetes

We have been running into an issue with the managed kubernetes cluster not deleting old docker images and filling up the disk space on the nodes causing new deployments to fail when pulling down new images casing an “No space left on device” error.

Reading into the kubernetes documentation the garbage collection should take care of deleting the unused images but this doesnt appear to be the case.

Is there a way to confirm the GC is running on each node and if it is not enable it?

Our current solution to resolve the problem is to setup a new pod that connect the the nodes docker instance and run the delete images command manually.

edited by AHA

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

2 answers

Hi there,

Could you open a support ticket providing the UUID of your cluster so we can look into this?

Alternatively, recycling the node should be a simple although “bandage” workaround.

We have the same problem, and we tried recycling the node, but after few days it gets full again. We even tried to upgrade the nodes but nothing works.

failed to garbage collect required amount of images. Wanted to free 16903334297 bytes, but freed 0 bytes

should we open a ticket?

Submit an Answer