Unable to deploy to Digital Ocean Kubernetes using GitLab CI

July 2, 2019 483 views
Kubernetes DigitalOcean
remcohaszing
By:
remcohaszing

I’m moving my review apps from Google Kubernetes to Digital Ocean Kubernetes.

I have succesfully linked the Kubernetes clusted to GitLab. I have installed Helm Tiller, Ingress, Cert-Manager, and Prometheus using the GitLab integration. I have upgraded Tiller to the latest version manually, as did I on the previous Google cluster.

The cluster is RBAC-enabled,GitLab-managed, configured to use the default namespace, and tied to the * wildcard environment scope.

Now my deployments are failing. job output

$ helm upgrade "$CI_ENVIRONMENT_SLUG" config/charts/appsemble-docs --atomic --install --set "git.commit.sha=$CI_COMMIT_SHA" --set "image.tag=$CI_COMMIT_REF_NAME"
Error: pods is forbidden: User "system:serviceaccount:default:default-service-account" cannot list resource "pods" in API group "" in the namespace "gitlab-managed-apps"

Manual deployments on my own laptop work fine.

What could be causing this issue?

Log In to Comment
1 Answer
jkwiatkoski July 2, 2019
Accepted Answer

Hi there,

This looks like an RBAC permissions issue. The error you’re seeing is saying the service-account “default-service-account” in namespace “default” doesnt have access to pods in the ‘gitlab-managed-apps’ namespace.

You can fix this by either creating a role that allows this access then binding the sa to that role witha rolebinding:

See below for examples to achieve something similar:

This will create a role pod-reader that can get, list, and watch pods

kubectl create role pod-reader –verb=get,list,watch –resource=pods -n <namespace to read pods>

Then we will create the rolebinding to associate a serviceaccount to this role that can view pods.

kubectl create rolebinding sa-read-pods –role=pod-reader –user=system:serviceaccount:default:default-service-account -n rbac

Let me know if you have any further questions.

Regards,

John Kwiatkoski
Senior Developer Support Engineer

Reply
  • remcohaszing July 2, 2019

    I deleted the cluster and created a new cluster. I changed the namespace in GitLab to appsemble instead, so the command for the role binding changed to:

    kubectl create rolebinding sa-read-pods --role=pod-reader --user=system:serviceaccount:appsemble:appsemble-service-account -n gitlab-managed-apps

    I also needed to be able to port forward, so I changed the role to this:

    apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: pod-reader
  namespace: gitlab-managed-apps
rules:
- apiGroups:
  - ""
  resources:
  - pods
  - pods/portforward
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - pods/portforward
  verbs:
  - create
  - delete
  - deletecollection
  - patch
  - update

    I do wonder, any clue why this is necessary for the cluster in Digital Ocean, but not for the one in Google Cloud?

  • remcohaszing July 2, 2019

    Thanks by the way! :D

Have another answer? Share your knowledge.

Related