Kubernetes volumes not working

The kubernetes CSI volume driver stopped working yesterday. Now we can’t provisioning any volumes what makes the managed kubernetes almost unusable.

For example, if we try to create a simple deplyoment wit a volume as described in your docs ( the volume doesn’t get provisioned.

Create first the pvc

apiVersion: v1
kind: PersistentVolumeClaim
  name: do-test
  - ReadWriteOnce
      storage: 5Gi
  storageClassName: do-block-storage

Then create the deployment

apiVersion: apps/v1 
kind: Deployment
  name: my-csi-app
  namespace: ioneaccess
      app: my-csi-app
  replicas: 1
        app: my-csi-app
        - name: my-frontend
          image: busybox
          - mountPath: "/data"
            name: my-do-volume
          command: [ "sleep", "1000000" ]
        - name: my-do-volume
            claimName: csi-test

After creating the PVC resource, the pvc stucks in pending state. And there are no log ouptuts from the csi-do-node pod which indicates that the driver tries or fail to create the volume.

1m          7m           26        do-test.158cc309ab1f4732                       PersistentVolumeClaim               Normal    ExternalProvisioning   persistentvolume-controller                                                           waiting for a volume to be created, either by external provisioner "" or manually created by system administrator
3m          7m           2         do-test.158cc309ac5b708a                       PersistentVolumeClaim               Normal    Provisioning           dobs.csi.digitalocean.com_master-ionedev-beook_e76a0359-480f-11e9-a030-229fa05727f2   External provisioner is provisioning volume for claim "ioneaccess/do-test"
21s         4m           2         do-test.158cc3392dbd9349                       PersistentVolumeClaim               Warning   ProvisioningFailed     dobs.csi.digitalocean.com_master-ionedev-beook_e76a0359-480f-11e9-a030-229fa05727f2   failed to provision volume with StorageClass "do-block-storage": timed out waiting for the condition

The volumes worked well for about 3 weeks. But after we had to recycle a kubernetes nodes due an unhealthy state, no more volumes working (except the ones which were provisioned and attached/mounted before this issue)

Anyone an idea?

Submit an answer

This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

Sign In or Sign Up to Answer

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

Want to learn more? Join the DigitalOcean Community!

Join our DigitalOcean community of over a million developers for free! Get help and share knowledge in Q&A, subscribe to topics of interest, and get courses and tools that will help you grow as a developer and scale your project or business.

We couldn’t solve the problem by ourself. But after the support restarted our master node, the volumes worked again. (The restart was required due another problem)

It seems that the DO API/Event issue also affects the volume mounting Since this morning we are unable to mount any volumes.

same for us here:

waiting for a volume to be created, either by external provisioner “” or manually created by system administrator