I'm having a pod in a deployment reliably fail with a "SandboxChanged" error
I’m deploying an application with a basic architecture and Helm chart. I had previously been trying to do a rolling deployment with a standard kubectl apply command, but the update would always fail, with new, dead containers continually created with every deployment.
In search of a solution, I shifted over to using Helm in the hopes that it would solve itself. I still have the problem but am at least seeing a new error.
The pod starts fine, but suffers a pull failure.
Failed to pull image "registry.gitlab.com/<user>/<name>:<rag>": rpc error: code = Unknown desc = Error response from daemon: Get https://registry.gitlab.com/v2/<user>/<name>/manifests/<tag>: unauthorized: HTTP Basic: Access denied
After the standard flow, I see a new error.
Pod sandbox changed, it will be killed and re-created.
After this, the standard
Error: ImagePullBackOff loop begins.
What does this error mean? I want to stress that two of the three pods start just fine, but one pod always fails. As such, I assume it is not a problem with my Gitlab auth information.
If I delete my deployment before doing the new deploy, everything works.