Related

Product Managed Kubernetes on DigitalOcean Product
Ingress rule cannot resolve the backend server ip address Question Question
Kubernetes PVC for daemonsets Question Question

Question

Gluster FS shared volumes for Kubernetes Cluster

Posted March 14, 2019 5.9k views
CentOSKubernetes

Hi,

Right now i am Testing DO Kubernetes and am starting to get my head around it.
There is only one thing that i am missing. An easy way to add shared volumes between PODS.

I tried GlusterFS since I am Familiar with it and Kubernetes has build in support for it, but so fare i Failed to get it working.

The GlusterFS is fine but I get an Error when I try to mound it.

“All subvolumes are down. Going offline until atleast one of them comes back up. The message "E [MSGID: 101075] [common-utils.c:294:gfresolveip6] 0-resolver: getaddrinfo failed (Name or service not known)”

I have read that Rook.io would be a alternative solution but am still hesitant.

Is there a way to use Gluster FS on DO Kubernetes ore are there any other solutions for my need Kind regards.

Add a comment
gradlon
By:
gradlon

Related

Product Managed Kubernetes on DigitalOcean Product
Ingress rule cannot resolve the backend server ip address Question Question
Kubernetes PVC for daemonsets Question Question

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
Submit an Answer
4 answers
gradlon March 25, 2019

For all those of you wondering.
I found the Solution.
One caveat, the endpoint has to be in the same namespace as the pod Consuming it.

  1. Create a Service for Gluster

    ---
apiVersion: v1
kind: Service
metadata:
name: glusterfs-cluster
namespace: global-storage
spec:
ports:
- port: 1

  2. Create endpoint for your Service

    ---
apiVersion: v1
kind: Endpoints
metadata:
name: glusterfs-cluster
namespace: default
subsets:
- addresses:
  - ip: 10.135.112.37
ports:
  - port: 1
- addresses:
  - ip: 10.135.53.65
ports:
  - port: 1
- addresses:
  - ip: 10.135.26.91
ports:
  - port: 1

  3. Consume Shared Volume

    ---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
namespace: default
labels:
app: nginx
spec:
replicas: 1
selector:
matchLabels:
  app: nginx
template:
metadata:
  labels:
    app: nginx
spec:
  containers:
  - name: nginx
    image: nginx:1.15.4
    ports:
    - containerPort: 80
    volumeMounts:
    - mountPath: /usr/share/nginx/html
      name: nfs-vol
    - mountPath: /data
      name: glusterfsvol
  volumes:
  - name: nfs-vol
    nfs:
      server: glusterfs.cloud.integral-systems.ch
      path: /globalshare
  - name: glusterfsvol
    glusterfs:
      endpoints: glusterfs-cluster
      path: globalshare
      readOnly: false

Hope this helps :)

Reply Report
milos1995 March 15, 2019

GlusterFS Native Storage Service for Kubernetes. gluster-kubernetes is a project to provide Kubernetes administrators a mechanism to easily deploy GlusterFS as a native storage service onto an existing Kubernetes cluster shareit apk.

Reply Report
liveb4buy May 15, 2019

@gradlon , Still its not working for me with the equivalent code (considering different namespace not default namespace) . Did you configure StorageClass or Init-Container in the Deployment level?
Pls let me know . Thanks in advance.

Reply Report
filiw January 7, 2021

When applying this test configuration:

---
apiVersion: v1
kind: Service
metadata:
  name: glusterfs-cluster
  namespace: default
spec:
  ports:
    - port: 1

---
apiVersion: v1
kind: Endpoints
metadata:
  name: glusterfs-cluster
  namespace: default
subsets:
  - addresses:
      - ip: 10.x.x.x # IP of a droplet running glusterfs server
    ports:
      - port: 1

---
apiVersion: v1
kind: Pod
metadata:
  name: glusterfs
spec:
  containers:
    - name: glusterfs
      image: nginx
      volumeMounts:
        - mountPath: "/mnt/user/test"
          name: glusterfsvol
  volumes:
    - name: glusterfsvol
      glusterfs:
        endpoints: glusterfs-cluster
        path: test
        readOnly: false

I get the following error:

mount: /var/lib/kubelet/pods/some-uuid/volumes/kubernetes.io~glusterfs/glusterfsvol: unknown filesystem type 'glusterfs'.

It looks like the kubernetes worker nodes don’t have glusterfs-client installed, henceforth the glusterfs mount type is non-existent.

Did anyone solve this yet?

Reply Report
  • garypen January 24, 2021

    I’d like an answer to this as well. I agree with your assessment.

    I think this workaround involves NFS somehow, but I still can’t see how you can use glusterfs on the worker nodes without glusterfs-client installed.

    Reply Report

Related

Looking for something else?

Ask a new question Search for more help