Question

One of the pods is always down when deploying kubernetes cluster

We are just trying out our first iteration on Kubernetes. I followed the docs here: https://docs.digitalocean.com/products/kubernetes/how-to/deploy-using-github-actions/ including the ingress load balancer: https://docs.digitalocean.com/products/kubernetes/how-to/add-load-balancers/

It finally somehow works, but for some reason one of the pods is always considered down (even though it’s running). https://ibb.co/Wvkt7S9

As per tutorial, I am using github actions:

# This is a basic workflow to help you get started with Actions

name: Master build

# Controls when the workflow will run
on:
  # Triggers the workflow on push or pull request events but only for the master branch
  push:
    branches: [ master ]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # This workflow contains a single job called "build"
  build:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest

    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
      - uses: actions/setup-java@v2
        with:
          distribution: 'zulu' # See 'Supported distributions' for available options
          java-version: '11'

      - name: Check out repository code
        uses: actions/checkout@v2

      - uses: actions/cache@v2
        with:
          path: |
            ~/.gradle/caches
            ~/.gradle/wrapper
          key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
          restore-keys: |
            ${{ runner.os }}-gradle-

      - name: Install doctl
        uses: digitalocean/action-doctl@v2
        with:
          token: ${{ secrets.DIGITALOCEAN_ACCESS_TOKEN }}

      - name: Assemble
        run: ./gradlew -Dgrails.env=prod --no-daemon assemble

      - name: Copy WAR file
        run: cp -p build/libs/priz-api-0.1.war docker/app.war

      - name: Build container image
        run: docker build -t registry.digitalocean.com/priz/priz-api:$(echo $GITHUB_SHA | head -c7) docker

      - name: Log in to DigitalOcean Container Registry with short-lived credentials
        run: doctl registry login --expiry-seconds 1200

      - name: Push image to DigitalOcean Container Registry
        run: docker push registry.digitalocean.com/priz/priz-api:$(echo $GITHUB_SHA | head -c7)

      - name: Set environment variables
        uses: danielr1996/envsubst-action@1.0.0
        env:
          PRIZ_DATABASE_URL_PROD: ${{ secrets.PRIZ_DATABASE_URL_PROD }}
          PRIZ_DATABASE_USER_PROD: ${{ secrets.PRIZ_DATABASE_USER_PROD }}
          PRIZ_DATABASE_PASSWORD_PROD: ${{ secrets.PRIZ_DATABASE_PASSWORD_PROD }}
          PRIZ_AUTH0_DOMAIN_PROD: ${{ secrets.PRIZ_AUTH0_DOMAIN_PROD }}
          PRIZ_AUTH0_API_DOMAIN_PROD: ${{ secrets.PRIZ_AUTH0_API_DOMAIN_PROD }}
          PRIZ_AUTH0_API_CLIENT_ID_PROD: ${{ secrets.PRIZ_AUTH0_API_CLIENT_ID_PROD }}
          PRIZ_AUTH0_API_CLIENT_SECRET_PROD: ${{ secrets.PRIZ_AUTH0_API_CLIENT_SECRET_PROD }}
          PRIZ_APP_BASE_URL_PROD: ${{ secrets.PRIZ_APP_BASE_URL_PROD }}
          PRIZ_STRIPE_API_KEY_SECRET_PROD: ${{ secrets.PRIZ_STRIPE_API_KEY_SECRET_PROD }}
          PRIZ_SEARCH_HOST_PROD: ${{ secrets.PRIZ_SEARCH_HOST_PROD }}
        with:
          input: deployment/_deployment.yml
          output: deployment/deployment.yml

      - name: Update deployment file
        run: TAG=$(echo $GITHUB_SHA | head -c7) && sed -i 's|<IMAGE>|registry.digitalocean.com/priz/priz-api:'${TAG}'|' $GITHUB_WORKSPACE/deployment/deployment.yml

      - name: Save DigitalOcean kubeconfig with short-lived credentials
        run: doctl kubernetes cluster kubeconfig save --expiry-seconds 600 priz-prod

      - name: Deploy to DigitalOcean Kubernetes
        run: kubectl apply -f $GITHUB_WORKSPACE/deployment/deployment.yml

      - name: Create ingress
        run: kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.34.1/deploy/static/provider/do/deploy.yaml

      - name: Verify deployment
        run: kubectl rollout status deployment/priz-api

      - name: Verify ingress
        run: kubectl get svc --namespace=ingress-nginx

and Kubernetes config:

apiVersion: v1
kind: Service
metadata:
  name: priz-api
spec:
  ports:
    - port: 80
      targetPort: 8080
  selector:
    app: priz-api
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: priz-api
  labels:
    app: priz-api
spec:
  # modify replicas according to your case
  replicas: 2
  selector:
    matchLabels:
      app: priz-api
  template:
    metadata:
      labels:
        app: priz-api
    spec:
      containers:
        - name: priz-api
          image: <IMAGE>
          env:
            - name: PRIZ_DATABASE_URL
              value: "${PRIZ_DATABASE_URL_PROD}"
            - name: PRIZ_DATABASE_USER
              value: "${PRIZ_DATABASE_USER_PROD}"
            - name: PRIZ_DATABASE_PASSWORD
              value: "${PRIZ_DATABASE_PASSWORD_PROD}"
            - name: PRIZ_AUTH0_DOMAIN
              value: "${PRIZ_AUTH0_DOMAIN_PROD}"
            - name: PRIZ_AUTH0_API_DOMAIN
              value: "${PRIZ_AUTH0_API_DOMAIN_PROD}"
            - name: PRIZ_AUTH0_API_CLIENT_ID
              value: "${PRIZ_AUTH0_API_CLIENT_ID_PROD}"
            - name: PRIZ_AUTH0_API_CLIENT_SECRET
              value: "${PRIZ_AUTH0_API_CLIENT_SECRET_PROD}"
            - name: PRIZ_APP_BASE_URL
              value: "${PRIZ_APP_BASE_URL_PROD}"
            - name: PRIZ_STRIPE_API_KEY_SECRET
              value: "${PRIZ_STRIPE_API_KEY_SECRET_PROD}"
            - name: PRIZ_SEARCH_HOST
              value: "${PRIZ_SEARCH_HOST_PROD}"
          ports:
            - containerPort: 8080
          resources:
            requests:
              cpu: 500m
              memory: 500Mi
            limits:
              cpu: 2000m
              memory: 2000Mi
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: priz-api-ingress
spec:
  rules:
    - host: apii.priz.guru
      http:
        paths:
          - backend:
              serviceName: priz-api
              servicePort: 80

Everything works and apii.riz.guru is accessible. But, again, one of the pods is considered down and ignored by LB. What am I missing?


Submit an answer

This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

Sign In or Sign Up to Answer

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

Hello,

I could suggest using the kubectl describe pod_name command to get more information on what the current state of the pod is.

Feel free to share the output here.

Also, it is usually a good idea to check the logs of the pod for more information too:

kubectl logs pod_name

Best,

Bobby