Using DO k8s container registry: authentication required

Question

I am trying to use registry.digitalocean.com for my k8s images.

I’ve created an image, tagged it, logged into the registry successfully (using doctl registry login) then pushed the image to the registry.

However, when starting the service it’s not creating pods successfully and giving :

Failed to pull image "registry.digitalocean.com/ordering-reg/mongodb-service": rpc error: code = Unknown desc = Error response from daemon
Get https://registry.digitalocean.com/v2/ordering-reg/mongodb-service/manifests/latest: unauthorized: authentication required

The pod status is “ImagePullBackOff” and it can’t pull the image.

Does anyone have any idea what might be causing this? My understanding was that if I was logged into the registry OK with doctl, then everything should be authed to pull the image from the registry.

Any help would be gratefully appreciated.

Answer 1

colinjohnriddell July 6, 2020

I had missed out a couple of steps in the guide DigitalOcean Container Registry Quickstart specifically the steps:

Use the kubernetes-manifest command to store an authentication token for your registry as a secret in your cluster:

doctl registry kubernetes-manifest | kubectl apply -f -
Copy
Instruct the default service account to use the authentication token as an imagePullSecret. Be sure to replace <my-registry> in the name field in the below command while keeping the registry- prefix:

kubectl patch serviceaccount default -p '{"imagePullSecrets": [{"name": "registry-<my-registry>"}]}'

Reply Report

Answer 2

zitro November 25, 2020

In my case, the problem was with “Download Docker Credentials” I used in the UI. I created a k8s secret using that downloaded files, and spent a couple of hours to finally dig out that doctl registry kubernetes-manifest gives another credentials. Surprisingly, the credentials downloaded from the UI were valid for pushing/pulling from the local machine, but not for pulling images inside the cluster. Strange!

Reply Report

Answer 3

billybaldisser August 14, 2020

I have follow all the steps and I am getting the same Authentication error. It is driving me crazy! Is someone else having the same problem?

Reply Report

Eltnen August 17, 2020

Having the same issue.
Strange thing is.

I first pulled an image successfully but my others fail.
The only thing I changed was the Namespace I am using.
I thought this caused the issue but I confirmed that both the secrets and serviceaccount are added to that namespace
Reply Report
- billybaldisser August 17, 2020
  
  Yes exactly, I did the same!
  in the end I discovered that the namespace is the problem!
  In fact, the command showed in the documentation is using the “default” namespace.
  I solved it by doing this:
  
  First get the registry yaml file
  
  doctl registry kubernetes-manifest > registry.yaml
  
  then change the namespace in the generated file to your namespace, and finally apply the new yaml file:
  
  kubectl -n NAMESPACE apply -f registry.yaml
  
  I hope it solve your problem!
  
  Reply Report
  
  Eltnen August 17, 2020
  
  Hey thanks for the response. Just figured it myself ....
  
  while in the default namespace I used
  
  edit secret registry-<my-repo>
  
  then I copied the contents and replaced all default namespaces to my own, and saved the file for later reference when I need more namespaces.
  
  I also applied the saved file as well to apply the secret to my namespace
  
  Reply Report
  
  billGiantSquid August 24, 2020
  
  Hello! How can I find my namespace to replace in the registry.yaml file?
  
  Reply Report
  
  Eltnen August 24, 2020
  
  if your run
  
  kubectl get namespaces
  
  you will see your current namespaces in kubernetes.
  If you never created a new one for your deployments that you are probably just on “default”.
  
  To see in what namespace you are operating right now you can run the following command:
  
  kubectl config view --minify | grep namespace:
  
  There are 2 ways to fix this issue but I eventually chose to do the following:
  
  create namespace to run my pods
  
  switch to that namespace
  
  create the pull secret in that namespace (if you follow the DO instructions it is automatically created in the default namespace). If you followed the guide you can probably run kubectl edit secret registry-<your-registry> -n default Copy the contents of the file and save them in a yaml file on your pc. In this file change the word DEFAULT to the name of your namespace and apply the file.
  
  Now you either have to set the pull secret as default for the service account in your namespace or you can add the secret to your deployments so that they know what secret to use. imagePullSecrets: - name: registry-<your-registry>
  
  If this is still a bit to vague just let me know.
  It took me some time to figure out as well.
  
  Reply Report
  
  billGiantSquid August 24, 2020
  
  Thank you so much! I’m at step 3 but when I tried applying the file with kubectl -n NAMESPACE apply -f registry.yaml I got for: "registry.yaml": Operation cannot be fulfilled on secrets: the object has been modified; please apply your changes to the latest version and try again
  
  Can you tell me how to apply it properly? Thank you!
  
  Reply Report
  
  billybaldisser August 25, 2020
  
  Hi, try removing “-n NAMESPACE”, I will fix my previous post.
  
  kubectl apply -f registry.yaml
  
  Sorry for that, it was my fault!
  
  Reply Report
colinjohnriddell August 17, 2020

Sorry to hear you’re having troubles. The two commands in my answer solved this for me :shrugs:
Reply Report
- billybaldisser August 17, 2020
  
  The thing is that serviceaccount is namespaced, so if you are trying to pull the images from a pod running inside a namespace, you have to add the secret to the default service account of that namespace.
  
  Reply Report

Answer 4

Eltnen August 17, 2020

Having the same issue.
Strange thing is.

I first pulled an image successfully but my others fail.
The only thing I changed was the Namespace I am using.
I thought this caused the issue but I confirmed that both the secrets and serviceaccount are added to that namespace
Reply Report
- billybaldisser August 17, 2020
  
  Yes exactly, I did the same!
  in the end I discovered that the namespace is the problem!
  In fact, the command showed in the documentation is using the “default” namespace.
  I solved it by doing this:
  
  First get the registry yaml file
  
  doctl registry kubernetes-manifest > registry.yaml
  
  then change the namespace in the generated file to your namespace, and finally apply the new yaml file:
  
  kubectl -n NAMESPACE apply -f registry.yaml
  
  I hope it solve your problem!
  
  Reply Report
  
  Eltnen August 17, 2020
  
  Hey thanks for the response. Just figured it myself ....
  
  while in the default namespace I used
  
  edit secret registry-<my-repo>
  
  then I copied the contents and replaced all default namespaces to my own, and saved the file for later reference when I need more namespaces.
  
  I also applied the saved file as well to apply the secret to my namespace
  
  Reply Report
  
  billGiantSquid August 24, 2020
  
  Hello! How can I find my namespace to replace in the registry.yaml file?
  
  Reply Report
  
  Eltnen August 24, 2020
  
  if your run
  
  kubectl get namespaces
  
  you will see your current namespaces in kubernetes.
  If you never created a new one for your deployments that you are probably just on “default”.
  
  To see in what namespace you are operating right now you can run the following command:
  
  kubectl config view --minify | grep namespace:
  
  There are 2 ways to fix this issue but I eventually chose to do the following:
  
  create namespace to run my pods
  
  switch to that namespace
  
  create the pull secret in that namespace (if you follow the DO instructions it is automatically created in the default namespace). If you followed the guide you can probably run kubectl edit secret registry-<your-registry> -n default Copy the contents of the file and save them in a yaml file on your pc. In this file change the word DEFAULT to the name of your namespace and apply the file.
  
  Now you either have to set the pull secret as default for the service account in your namespace or you can add the secret to your deployments so that they know what secret to use. imagePullSecrets: - name: registry-<your-registry>
  
  If this is still a bit to vague just let me know.
  It took me some time to figure out as well.
  
  Reply Report
  
  billGiantSquid August 24, 2020
  
  Thank you so much! I’m at step 3 but when I tried applying the file with kubectl -n NAMESPACE apply -f registry.yaml I got for: "registry.yaml": Operation cannot be fulfilled on secrets: the object has been modified; please apply your changes to the latest version and try again
  
  Can you tell me how to apply it properly? Thank you!
  
  Reply Report
  
  billybaldisser August 25, 2020
  
  Hi, try removing “-n NAMESPACE”, I will fix my previous post.
  
  kubectl apply -f registry.yaml
  
  Sorry for that, it was my fault!
  
  Reply Report
colinjohnriddell August 17, 2020

Sorry to hear you’re having troubles. The two commands in my answer solved this for me :shrugs:
Reply Report
- billybaldisser August 17, 2020
  
  The thing is that serviceaccount is namespaced, so if you are trying to pull the images from a pod running inside a namespace, you have to add the secret to the default service account of that namespace.
  
  Reply Report

Related

Question

Using DO k8s container registry: authentication required

Related

Leave a comment

Looking for something else?

Sign up for our newsletter

Related

Question

Using DO k8s container registry: authentication required

Related

Leave a comment

Related

question

Kubernetes cluster outages

question

Kubernetes: How do I access the CA to sign a new user certificate?

question

kubectl with managed certificate not working

question

How do you re-attach a persistent volume to a kubernetes cluster?

Looking for something else?