Question

Gitlab Registry Forbidden with managed Kubernetes Cluster

Posted September 16, 2019 1.4k views
DigitalOcean Kubernetes DigitalOcean Managed Kubernetes

Hello,

I’m currently trying out the managed Kubernetes Cluster with the Gitlab CI/CD and integrated private Registry. To understand my problem you need to know that I currently have a working Docker Swarm Cluster with that CI/CD and private Registry in use.

However with the managed Kubernetes the Pod Creation is always stuck at “ImagePullBackOff” with following error:

Failed to pull image "registry.gitlab.com/<PROJECT-PATH>/<BRANCH>:<COMMIT HASH>": rpc error: code = Unknown desc = Error response from daemon: Get "registry.gitlab.com/<PROJECT-PATH>/<BRANCH>:<COMMIT HASH>": denied: access forbidden

I tried it with Helm/Tiller and also with a normal Kubernetes Deploy file. The secret is in the same namespace as the deployment and is working on the mentioned Docker Swarm and on my local machine.

Is DO doing something weird here? Maybe someone of you have more information.

Kind regards,
Kai

edited by AHA
1 comment

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

4 answers

You need to create a secret to authorize kubernetes to pull images from the registry.

In order to do that you may need to create a Secret Object with the base64 of your local dockerconfig.json like so:

apiVersion: v1
kind: Secret
metadata:
  name: docker-registry-configuration
  namespace: your-namespace
data:
  .dockerconfigjson: base64encodedstring
type: kubernetes.io/dockerconfigjson

---


apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: your-application
  namespace: your-namespace
spec:
  template:
    spec:
      containers:
        image: myprivate.registry.com/image-name
        name: container-name
      imagePullSecrets:
      - name: docker-registry-configuration
      restartPolicy: Always
status: {}

Hi,

as mentioned I have a secret in place for this image. It’s a Gitlab Deploy Token which is pushed to the kubernetes like this:

kubectl create secret -n "$KUBE_NAMESPACE" \
    docker-registry gitlab-registry \
    --docker-server="$CI_REGISTRY" \
    --docker-username="$CI_REGISTRY_USER" \
    --docker-password="$CI_REGISTRY_PASSWORD" \
    --docker-email="$GITLAB_USER_EMAIL" \
    -o yaml --dry-run | kubectl replace -n "$KUBE_NAMESPACE" --force -f -

The Variables here are environment Variables in the Gitlab CI.

After creating the secret I deploy the application with helm:

if [[ -z "$CI_COMMIT_TAG" ]]; then
    image_repository=${CI_APPLICATION_REPOSITORY:-$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG}
    image_tag=${CI_APPLICATION_TAG:-$CI_COMMIT_SHA}
  else
    image_repository=${CI_APPLICATION_REPOSITORY:-$CI_REGISTRY_IMAGE}
    image_tag=${CI_APPLICATION_TAG:-$CI_COMMIT_TAG}
  fi


helm upgrade --install \
      --wait \
      --set service.enabled="$service_enabled" \
      --set gitlab.app="$CI_PROJECT_PATH_SLUG" \
      --set gitlab.env="$CI_ENVIRONMENT_SLUG" \
      --set gitlab.envName="$CI_ENVIRONMENT_NAME" \
      --set gitlab.envURL="$CI_ENVIRONMENT_URL" \
      --set releaseOverride="$RELEASE_NAME" \
      --set image.repository="$image_repository" \
      --set image.tag="$image_tag" \
      --set image.pullPolicy=IfNotPresent \
      --set image.secrets[0].name="$secret_name" \
      --set application.track="$track" \
      --set application.database_url="$DATABASE_URL" \
      --set application.secretName="$APPLICATION_SECRET_NAME" \
      --set application.secretChecksum="$APPLICATION_SECRET_CHECKSUM" \
      --set service.commonName="le-$CI_PROJECT_ID.$KUBE_INGRESS_BASE_DOMAIN" \
      --set service.url="$CI_ENVIRONMENT_URL" \
      --set service.additionalHosts="$additional_hosts" \
      --set replicaCount="$replicas" \
      --set application.migrateCommand="$DB_MIGRATE" \
      $HELM_UPGRADE_EXTRA_ARGS \
      --namespace="$KUBE_NAMESPACE" \
      "$name" \
      chart/

Any movement or insight on this?

I had the same issue but in my case was a typo on the secret namespace. When I fixed that my problem got solved.

Submit an Answer