Hi,
My Kubernetes operators are unable to spin up any Pods because they are not able to mount persistent volumes (Block storage from Digital Ocean) and Pods remain in PENDING condition forever.
I am simply following this tutorial from Redis blog.
Here are my steps:
hostname:~$ kubectl create namespace redis
namespace/redis created
hostname:~$ kubectl config set-context --current --namespace=redis
Context "do-nyc3-k8s-1-20-2-do-0-nyc3-1623035038930" modified.
hostname:~$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
pool-yz2m3trll-8e25f Ready <none> 2m2s v1.20.2
pool-yz2m3trll-8e25x Ready <none> 2m9s v1.20.2
pool-yz2m3trll-8e25y Ready <none> 2m44s v1.20.2
hostname:~$ kubectl get storageClass
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
do-block-storage (default) dobs.csi.digitalocean.com Delete Immediate true 5m27s
hostname:~$ kubectl get pvc
No resources found in default namespace.
hostname:~$ kubectl get pv
No resources found
hostname:~$ VERSION=`curl --silent https://api.github.com/repos/RedisLabs/redis-enterprise-k8s-docs/releases/latest | grep tag_name | awk -F'"' '{print $4}'`
hostname:~$ curl --silent -O https://raw.githubusercontent.com/RedisLabs/redis-enterprise-k8s-docs/$VERSION/bundle.yaml
hostname:~$ kubectl apply -f bundle.yaml
role.rbac.authorization.k8s.io/redis-enterprise-operator created
rolebinding.rbac.authorization.k8s.io/redis-enterprise-operator created
serviceaccount/redis-enterprise-operator created
Warning: apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
customresourcedefinition.apiextensions.k8s.io/redisenterpriseclusters.app.redislabs.com created
deployment.apps/redis-enterprise-operator created
customresourcedefinition.apiextensions.k8s.io/redisenterprisedatabases.app.redislabs.com created
hostname:~$ kubectl get all --namespace=redis
NAME READY STATUS RESTARTS AGE
pod/redis-enterprise-operator-85d9f666fb-57gpd 1/1 Running 0 38s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/redis-enterprise-operator 1/1 1 1 42s
NAME DESIRED CURRENT READY AGE
replicaset.apps/redis-enterprise-operator-85d9f666fb 1 1 1 42s
hostname:~$ cat <<EOF | kubectl apply -f -
apiVersion: "app.redislabs.com/v1"
kind: "RedisEnterpriseCluster"
metadata:
name: "test-cluster"
spec:
nodes: 3
persistentSpec:
enabled: true
storageClassName: "do-block-storage"
volumeSize: "4Gi" #optional
redisEnterpriseNodeResources:
limits:
cpu: 2000m
memory: 1.4Gi
requests:
cpu: 2000m
memory: 1.4Gi
EOF
redisenterprisecluster.app.redislabs.com/test-cluster created
hostname:~$ kubectl get all --namespace=redis
NAME READY STATUS RESTARTS AGE
pod/redis-enterprise-operator-85d9f666fb-57gpd 1/1 Running 0 6m53s
pod/test-cluster-services-rigger-869687b654-x4tmf 1/1 Running 0 3m54s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/test-cluster ClusterIP None <none> 9443/TCP,8001/TCP,8070/TCP 3m56s
service/test-cluster-ui ClusterIP 10.245.79.231 <none> 8443/TCP 3m56s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/redis-enterprise-operator 1/1 1 1 6m57s
deployment.apps/test-cluster-services-rigger 1/1 1 1 3m58s
NAME DESIRED CURRENT READY AGE
replicaset.apps/redis-enterprise-operator-85d9f666fb 1 1 1 6m58s
replicaset.apps/test-cluster-services-rigger-869687b654 1 1 1 3m59s
NAME READY AGE
statefulset.apps/test-cluster 0/3 4m
hostname:~$ kubectl get rec
NAME NODES VERSION STATE SPEC STATUS LICENSE STATE SHARDS LIMIT LICENSE EXPIRATION DATE AGE
test-cluster 3 6.0.20-69 BootstrappingFirstPod Valid 67m
I deployed the test cluster with instructions provided here: https://docs.redislabs.com/latest/platforms/kubernetes/concepts/persistent-volumes/
You can see the cluster remains in BootstrappingFirstPod state forever. What am I missing?
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
Hi there,
When Kubernetes operators are unable to mount persistent volumes and Pods remain in a PENDING state, there are several potential issues to investigate:
Check Access Modes:
ReadWriteOnce access mode, as DigitalOcean Block Storage doesn’t support ReadOnlyMany or ReadWriteMany.kubectl describe pvc -n redis to inspect the PVCs and check the AccessModes field.Inspect Storage Class:
do-block-storage Storage Class is set as the default and is properly configured to dynamically provision storage with the correct access mode.kubectl describe sc do-block-storage to view the parameters and ensure that the provisioner is set to dobs.csi.digitalocean.com.Check PVC Binding:
kubectl get pvc -n redis to ensure that PVCs are in a Bound state and not stuck in Pending.kubectl describe pvc PVC_NAME -n redis to identify any errors.Best,
Bobby
Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.
Full documentation for every DigitalOcean product.
The Wave has everything you need to know about building a business, from raising funding to marketing your product.
Stay up to date by signing up for DigitalOcean’s Infrastructure as a Newsletter.
New accounts only. By submitting your email you agree to our Privacy Policy
Scale up as you grow — whether you're running one virtual machine or ten thousand.
Sign up and get $200 in credit for your first 60 days with DigitalOcean.*
*This promotional offer applies to new accounts only.