I'm having a pod in a deployment reliably fail with a "SandboxChanged" error

Posted November 14, 2019 3k views
DigitalOcean Managed Kubernetes

I’m deploying an application with a basic architecture and Helm chart. I had previously been trying to do a rolling deployment with a standard kubectl apply command, but the update would always fail, with new, dead containers continually created with every deployment.

In search of a solution, I shifted over to using Helm in the hopes that it would solve itself. I still have the problem but am at least seeing a new error.

The pod starts fine, but suffers a pull failure.

Failed to pull image "<user>/<name>:<rag>": rpc error: code = Unknown desc = Error response from daemon: Get<user>/<name>/manifests/<tag>: unauthorized: HTTP Basic: Access denied

After the standard flow, I see a new error.

Pod sandbox changed, it will be killed and re-created.

After this, the standard Error: ImagePullBackOff loop begins.

What does this error mean? I want to stress that two of the three pods start just fine, but one pod always fails. As such, I assume it is not a problem with my Gitlab auth information.

If I delete my deployment before doing the new deploy, everything works.

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

Submit an Answer
1 answer

Did you enabled the sandbox on the nodes , if yes then there might be some restart of the demon set that helps on managing the system level calls in the nodes.
check the following doc for more info .

Its the sandbox implementation on GCP cloud.