Gitlab Registry Forbidden with managed Kubernetes Cluster

September 16, 2019 199 views
Kubernetes DigitalOcean

Hello,

I’m currently trying out the managed Kubernetes Cluster with the Gitlab CI/CD and integrated private Registry. To understand my problem you need to know that I currently have a working Docker Swarm Cluster with that CI/CD and private Registry in use.

However with the managed Kubernetes the Pod Creation is always stuck at “ImagePullBackOff” with following error:

Failed to pull image "registry.gitlab.com/<PROJECT-PATH>/<BRANCH>:<COMMIT HASH>": rpc error: code = Unknown desc = Error response from daemon: Get "registry.gitlab.com/<PROJECT-PATH>/<BRANCH>:<COMMIT HASH>": denied: access forbidden

I tried it with Helm/Tiller and also with a normal Kubernetes Deploy file. The secret is in the same namespace as the deployment and is working on the mentioned Docker Swarm and on my local machine.

Is DO doing something weird here? Maybe someone of you have more information.

Kind regards,
Kai

1 comment
2 Answers

You need to create a secret to authorize kubernetes to pull images from the registry.

In order to do that you may need to create a Secret Object with the base64 of your local dockerconfig.json like so:

apiVersion: v1
kind: Secret
metadata:
  name: docker-registry-configuration
  namespace: your-namespace
data:
  .dockerconfigjson: base64encodedstring
type: kubernetes.io/dockerconfigjson

---


apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: your-application
  namespace: your-namespace
spec:
  template:
    spec:
      containers:
        image: myprivate.registry.com/image-name
        name: container-name
      imagePullSecrets:
      - name: docker-registry-configuration
      restartPolicy: Always
status: {}

Hi,

as mentioned I have a secret in place for this image. It’s a Gitlab Deploy Token which is pushed to the kubernetes like this:

kubectl create secret -n "$KUBE_NAMESPACE" \
    docker-registry gitlab-registry \
    --docker-server="$CI_REGISTRY" \
    --docker-username="$CI_REGISTRY_USER" \
    --docker-password="$CI_REGISTRY_PASSWORD" \
    --docker-email="$GITLAB_USER_EMAIL" \
    -o yaml --dry-run | kubectl replace -n "$KUBE_NAMESPACE" --force -f -

The Variables here are environment Variables in the Gitlab CI.

After creating the secret I deploy the application with helm:

if [[ -z "$CI_COMMIT_TAG" ]]; then
    image_repository=${CI_APPLICATION_REPOSITORY:-$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG}
    image_tag=${CI_APPLICATION_TAG:-$CI_COMMIT_SHA}
  else
    image_repository=${CI_APPLICATION_REPOSITORY:-$CI_REGISTRY_IMAGE}
    image_tag=${CI_APPLICATION_TAG:-$CI_COMMIT_TAG}
  fi


helm upgrade --install \
      --wait \
      --set service.enabled="$service_enabled" \
      --set gitlab.app="$CI_PROJECT_PATH_SLUG" \
      --set gitlab.env="$CI_ENVIRONMENT_SLUG" \
      --set gitlab.envName="$CI_ENVIRONMENT_NAME" \
      --set gitlab.envURL="$CI_ENVIRONMENT_URL" \
      --set releaseOverride="$RELEASE_NAME" \
      --set image.repository="$image_repository" \
      --set image.tag="$image_tag" \
      --set image.pullPolicy=IfNotPresent \
      --set image.secrets[0].name="$secret_name" \
      --set application.track="$track" \
      --set application.database_url="$DATABASE_URL" \
      --set application.secretName="$APPLICATION_SECRET_NAME" \
      --set application.secretChecksum="$APPLICATION_SECRET_CHECKSUM" \
      --set service.commonName="le-$CI_PROJECT_ID.$KUBE_INGRESS_BASE_DOMAIN" \
      --set service.url="$CI_ENVIRONMENT_URL" \
      --set service.additionalHosts="$additional_hosts" \
      --set replicaCount="$replicas" \
      --set application.migrateCommand="$DB_MIGRATE" \
      $HELM_UPGRADE_EXTRA_ARGS \
      --namespace="$KUBE_NAMESPACE" \
      "$name" \
      chart/

Have another answer? Share your knowledge.