Question

Gluster FS shared volumes for Kubernetes Cluster

Hi,

Right now i am Testing DO Kubernetes and am starting to get my head around it. There is only one thing that i am missing. An easy way to add shared volumes between PODS.

I tried GlusterFS since I am Familiar with it and Kubernetes has build in support for it, but so fare i Failed to get it working.

The GlusterFS is fine but I get an Error when I try to mound it.

"All subvolumes are down. Going offline until atleast one of them comes back up. The message “E [MSGID: 101075] [common-utils.c:294:gf_resolve_ip6] 0-resolver: getaddrinfo failed (Name or service not known)”

I have read that Rook.io would be a alternative solution but am still hesitant.

Is there a way to use Gluster FS on DO Kubernetes ore are there any other solutions for my need Kind regards.


Submit an answer


This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

Sign In or Sign Up to Answer

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

Accepted Answer

For all those of you wondering. I found the Solution. One caveat, the endpoint has to be in the same namespace as the pod Consuming it.

  1. Create a Service for Gluster
---
apiVersion: v1
kind: Service
metadata:
  name: glusterfs-cluster
  namespace: global-storage
spec:
  ports:
  - port: 1
  1. Create endpoint for your Service
---
apiVersion: v1
kind: Endpoints
metadata:
  name: glusterfs-cluster
  namespace: default
subsets:
  - addresses:
      - ip: 10.135.112.37
    ports:
      - port: 1
  - addresses:
      - ip: 10.135.53.65
    ports:
      - port: 1
  - addresses:
      - ip: 10.135.26.91
    ports:
      - port: 1
  1. Consume Shared Volume
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  namespace: default
  labels:
    app: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.15.4
        ports:
        - containerPort: 80
        volumeMounts:
        - mountPath: /usr/share/nginx/html
          name: nfs-vol
        - mountPath: /data
          name: glusterfsvol
      volumes:
      - name: nfs-vol
        nfs:
          server: glusterfs.cloud.integral-systems.ch
          path: /globalshare
      - name: glusterfsvol
        glusterfs:
          endpoints: glusterfs-cluster
          path: globalshare
          readOnly: false

Hope this helps :)

When applying this test configuration:

---
apiVersion: v1
kind: Service
metadata:
  name: glusterfs-cluster
  namespace: default
spec:
  ports:
    - port: 1

---
apiVersion: v1
kind: Endpoints
metadata:
  name: glusterfs-cluster
  namespace: default
subsets:
  - addresses:
      - ip: 10.x.x.x # IP of a droplet running glusterfs server
    ports:
      - port: 1

---
apiVersion: v1
kind: Pod
metadata:
  name: glusterfs
spec:
  containers:
    - name: glusterfs
      image: nginx
      volumeMounts:
        - mountPath: "/mnt/user/test"
          name: glusterfsvol
  volumes:
    - name: glusterfsvol
      glusterfs:
        endpoints: glusterfs-cluster
        path: test
        readOnly: false

I get the following error:

mount: /var/lib/kubelet/pods/some-uuid/volumes/kubernetes.io~glusterfs/glusterfsvol: unknown filesystem type 'glusterfs'.

It looks like the kubernetes worker nodes don’t have glusterfs-client installed, henceforth the glusterfs mount type is non-existent.

Did anyone solve this yet?

@gradlon , Still its not working for me with the equivalent code (considering different namespace not default namespace) . Did you configure StorageClass or Init-Container in the Deployment level? Pls let me know . Thanks in advance.

Try DigitalOcean for free

Click below to sign up and get $200 of credit to try our products over 60 days!

Sign up

Featured on Community

Get our biweekly newsletter

Sign up for Infrastructure as a Newsletter.

Hollie's Hub for Good

Working on improving health and education, reducing inequality, and spurring economic growth? We'd like to help.

Become a contributor

Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.

Welcome to the developer cloud

DigitalOcean makes it simple to launch in the cloud and scale up as you grow — whether you're running one virtual machine or ten thousand.

Learn more