Advanced Django CI/CD Pipeline with GitHub Actions

Part 1: Project Setup

1.1 Project Structure

your-django-project/
├── .github/
│   └── workflows/
│       ├── ci.yml
│       ├── cd-staging.yml
│       └── cd-production.yml
├── app/
├── config/
├── requirements/
│   ├── base.txt
│   ├── development.txt
│   ├── production.txt
│   └── testing.txt
├── tests/
├── manage.py
├── pytest.ini
├── .coveragerc
└── docker-compose.yml

1.2 Requirements Files

requirements/base.txt

Django>=4.2,<5.0
psycopg2-binary>=2.9
python-decouple>=3.8
gunicorn>=21.2
whitenoise>=6.6

requirements/testing.txt

-r base.txt
pytest>=7.4
pytest-django>=4.5
pytest-cov>=4.1
factory-boy>=3.3
faker>=20.0
coverage>=7.3

requirements/development.txt

-r testing.txt
black>=23.11
flake8>=6.1
isort>=5.12
pylint>=3.0
django-debug-toolbar>=4.2

1.3 Environment Variables Setup

Create .env.example:

DEBUG=False
SECRET_KEY=your-secret-key-here
DATABASE_URL=postgresql://user:password@localhost:5432/dbname
ALLOWED_HOSTS=localhost,127.0.0.1
DJANGO_SETTINGS_MODULE=config.settings.production

---

Part 2: Continuous Integration (CI)

2.1 Main CI Workflow

Create .github/workflows/ci.yml:

name: Django CI

on:
  push:
    branches: [ main, develop ]
  pull_request:
    branches: [ main, develop ]

jobs:
  test:
    name: Test Suite
    runs-on: ubuntu-latest

    services:
      postgres:
        image: postgres:15
        env:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: test_db
        ports:
          - 5432:5432
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5

      redis:
        image: redis:7
        ports:
          - 6379:6379
        options: >-
          --health-cmd "redis-cli ping"
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5

    strategy:
      matrix:
        python-version: ['3.10', '3.11', '3.12']

    steps:
    - name: Checkout code
      uses: actions/checkout@v4

    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v4
      with:
        python-version: ${{ matrix.python-version }}
        cache: 'pip'

    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements/testing.txt

    - name: Run migrations
      env:
        DATABASE_URL: postgresql://postgres:postgres@localhost:5432/test_db
        SECRET_KEY: test-secret-key-for-ci
        DEBUG: 'True'
      run: |
        python manage.py migrate --no-input

    - name: Run tests with coverage
      env:
        DATABASE_URL: postgresql://postgres:postgres@localhost:5432/test_db
        SECRET_KEY: test-secret-key-for-ci
        DEBUG: 'True'
      run: |
        pytest --cov=. --cov-report=xml --cov-report=html --cov-report=term

    - name: Upload coverage to Codecov
      uses: codecov/codecov-action@v3
      with:
        file: ./coverage.xml
        fail_ci_if_error: false

    - name: Archive coverage report
      uses: actions/upload-artifact@v3
      with:
        name: coverage-report
        path: htmlcov/

  lint:
    name: Code Quality
    runs-on: ubuntu-latest

    steps:
    - name: Checkout code
      uses: actions/checkout@v4

    - name: Set up Python
      uses: actions/setup-python@v4
      with:
        python-version: '3.11'
        cache: 'pip'

    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements/development.txt

    - name: Run Black
      run: black --check .

    - name: Run isort
      run: isort --check-only --profile black .

    - name: Run flake8
      run: flake8 . --max-line-length=88 --extend-ignore=E203,W503

    - name: Run pylint
      run: pylint **/*.py --disable=C0111,R0903 || true

  security:
    name: Security Checks
    runs-on: ubuntu-latest

    steps:
    - name: Checkout code
      uses: actions/checkout@v4

    - name: Set up Python
      uses: actions/setup-python@v4
      with:
        python-version: '3.11'

    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install safety bandit
        pip install -r requirements/base.txt

    - name: Run Safety check
      run: safety check --json || true

    - name: Run Bandit
      run: bandit -r . -f json -o bandit-report.json || true

    - name: Upload security reports
      uses: actions/upload-artifact@v3
      with:
        name: security-reports
        path: |
          bandit-report.json

  build:
    name: Build Check
    runs-on: ubuntu-latest
    needs: [test, lint, security]

    steps:
    - name: Checkout code
      uses: actions/checkout@v4

    - name: Set up Python
      uses: actions/setup-python@v4
      with:
        python-version: '3.11'

    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements/production.txt

    - name: Collect static files
      env:
        SECRET_KEY: test-secret-key
        DEBUG: 'False'
        DATABASE_URL: sqlite:///db.sqlite3
      run: |
        python manage.py collectstatic --no-input

    - name: Check deployment settings
      env:
        SECRET_KEY: test-secret-key
        DEBUG: 'False'
      run: |
        python manage.py check --deploy --fail-level WARNING

2.2 Pytest Configuration

Create pytest.ini:

[pytest]
DJANGO_SETTINGS_MODULE = config.settings.testing
python_files = tests.py test_*.py *_tests.py
addopts =
    --strict-markers
    --strict-config
    --cov=.
    --cov-branch
    --cov-report=term-missing:skip-covered
    --cov-report=html
    --cov-report=xml
    --no-cov-on-fail
    -ra
    --showlocals
    --tb=short
testpaths = tests
markers =
    slow: marks tests as slow
    integration: marks tests as integration tests
    unit: marks tests as unit tests

2.3 Coverage Configuration

Create .coveragerc:

[run]
source = .
omit =
    */migrations/*
    */tests/*
    */test_*.py
    */__pycache__/*
    */venv/*
    */virtualenv/*
    manage.py
    */wsgi.py
    */asgi.py

[report]
exclude_lines =
    pragma: no cover
    def __repr__
    raise AssertionError
    raise NotImplementedError
    if __name__ == .__main__.:
    if TYPE_CHECKING:
    @abstractmethod

---

Part 3: Continuous Deployment (CD)

3.1 Staging Deployment

Create .github/workflows/cd-staging.yml:

name: Deploy to Staging

on:
  push:
    branches: [ develop ]
  workflow_dispatch:

jobs:
  deploy-staging:
    name: Deploy to Staging Environment
    runs-on: ubuntu-latest
    environment:
      name: staging
      url: https://your-app-staging.railway.app

    steps:
    - name: Checkout code
      uses: actions/checkout@v4

    - name: Deploy to Railway (Staging)
      uses: bervProject/railway-deploy@main
      with:
        railway_token: ${{ secrets.RAILWAY_TOKEN_STAGING }}
        service: your-app-staging

    - name: Wait for deployment
      run: sleep 30

    - name: Run database migrations
      run: |
        # This depends on your platform's CLI
        railway run python manage.py migrate --no-input
      env:
        RAILWAY_TOKEN: ${{ secrets.RAILWAY_TOKEN_STAGING }}

    - name: Health check
      run: |
        curl --fail https://your-app-staging.railway.app/health/ || exit 1

    - name: Notify deployment
      if: always()
      uses: 8398a7/action-slack@v3
      with:
        status: ${{ job.status }}
        text: 'Staging deployment ${{ job.status }}'
        webhook_url: ${{ secrets.SLACK_WEBHOOK }}

3.2 Production Deployment

Create .github/workflows/cd-production.yml:

name: Deploy to Production

on:
  push:
    branches: [ main ]
    tags:
      - 'v*'
  workflow_dispatch:

jobs:
  deploy-production:
    name: Deploy to Production
    runs-on: ubuntu-latest
    environment:
      name: production
      url: https://your-app.com

    steps:
    - name: Checkout code
      uses: actions/checkout@v4

    - name: Create deployment
      uses: chrnorm/deployment-action@v2
      id: deployment
      with:
        token: ${{ secrets.GITHUB_TOKEN }}
        environment: production

    - name: Deploy to Railway (Production)
      uses: bervProject/railway-deploy@main
      with:
        railway_token: ${{ secrets.RAILWAY_TOKEN_PRODUCTION }}
        service: your-app-production

    - name: Wait for deployment
      run: sleep 60

    - name: Run database migrations
      run: |
        railway run python manage.py migrate --no-input
      env:
        RAILWAY_TOKEN: ${{ secrets.RAILWAY_TOKEN_PRODUCTION }}

    - name: Collect static files
      run: |
        railway run python manage.py collectstatic --no-input
      env:
        RAILWAY_TOKEN: ${{ secrets.RAILWAY_TOKEN_PRODUCTION }}

    - name: Health check
      run: |
        curl --fail https://your-app.com/health/ || exit 1

    - name: Update deployment status (success)
      if: success()
      uses: chrnorm/deployment-status@v2
      with:
        token: ${{ secrets.GITHUB_TOKEN }}
        deployment-id: ${{ steps.deployment.outputs.deployment_id }}
        state: success
        environment-url: https://your-app.com

    - name: Update deployment status (failure)
      if: failure()
      uses: chrnorm/deployment-status@v2
      with:
        token: ${{ secrets.GITHUB_TOKEN }}
        deployment-id: ${{ steps.deployment.outputs.deployment_id }}
        state: failure

    - name: Notify deployment
      if: always()
      uses: 8398a7/action-slack@v3
      with:
        status: ${{ job.status }}
        text: 'Production deployment ${{ job.status }}'
        webhook_url: ${{ secrets.SLACK_WEBHOOK }}

    - name: Create Sentry release
      if: success()
      uses: getsentry/action-release@v1
      env:
        SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_AUTH_TOKEN }}
        SENTRY_ORG: your-org
        SENTRY_PROJECT: your-project
      with:
        environment: production

---

Part 4: Advanced Features

4.1 Caching Dependencies

Speed up your workflows with caching:

    - name: Cache pip packages
      uses: actions/cache@v3
      with:
        path: ~/.cache/pip
        key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements*.txt') }}
        restore-keys: |
          ${{ runner.os }}-pip-

4.2 Matrix Testing with Multiple Databases

yaml

    strategy:
      matrix:
        python-version: ['3.10', '3.11', '3.12']
        database: ['postgresql', 'mysql']
        include:
          - database: postgresql
            db_port: 5432
            db_image: postgres:15
          - database: mysql
            db_port: 3306
            db_image: mysql:8

4.3 Parallel Testing

    - name: Run tests in parallel
      run: |
        pytest -n auto --dist loadscope

4.4 Docker Build and Push

  docker:
    name: Build and Push Docker Image
    runs-on: ubuntu-latest
    needs: [test, lint]

    steps:
    - name: Checkout code
      uses: actions/checkout@v4

    - name: Set up Docker Buildx
      uses: docker/setup-buildx-action@v3

    - name: Log in to Docker Hub
      uses: docker/login-action@v3
      with:
        username: ${{ secrets.DOCKER_USERNAME }}
        password: ${{ secrets.DOCKER_PASSWORD }}

    - name: Extract metadata
      id: meta
      uses: docker/metadata-action@v5
      with:
        images: yourusername/django-app
        tags: |
          type=ref,event=branch
          type=semver,pattern={{version}}
          type=sha

    - name: Build and push
      uses: docker/build-push-action@v5
      with:
        context: .
        push: true
        tags: ${{ steps.meta.outputs.tags }}
        labels: ${{ steps.meta.outputs.labels }}
        cache-from: type=registry,ref=yourusername/django-app:buildcache
        cache-to: type=registry,ref=yourusername/django-app:buildcache,mode=max

---

Part 5: GitHub Secrets Configuration

Set up these secrets in your GitHub repository (Settings → Secrets and variables → Actions):

Required Secrets:

• RAILWAY_TOKEN_STAGING - Railway token for staging
• RAILWAY_TOKEN_PRODUCTION - Railway token for production
• DOCKER_USERNAME - Docker Hub username
• DOCKER_PASSWORD - Docker Hub password/token
• SLACK_WEBHOOK - Slack webhook for notifications
• SENTRY_AUTH_TOKEN - Sentry authentication token
• CODECOV_TOKEN - Codecov token for coverage reports

---

Part 6: Health Check Endpoint

Create a health check view in your Django app:

# app/views.py
from django.http import JsonResponse
from django.db import connection
from django.core.cache import cache
import redis

def health_check(request):
    """
    Health check endpoint for CI/CD pipelines
    """
    health_status = {
        'status': 'healthy',
        'checks': {}
    }

    # Database check
    try:
        with connection.cursor() as cursor:
            cursor.execute("SELECT 1")
        health_status['checks']['database'] = 'ok'
    except Exception as e:
        health_status['status'] = 'unhealthy'
        health_status['checks']['database'] = f'error: {str(e)}'

    # Cache check
    try:
        cache.set('health_check', 'ok', 10)
        cache.get('health_check')
        health_status['checks']['cache'] = 'ok'
    except Exception as e:
        health_status['status'] = 'unhealthy'
        health_status['checks']['cache'] = f'error: {str(e)}'

    status_code = 200 if health_status['status'] == 'healthy' else 503
    return JsonResponse(health_status, status=status_code)

Add to your URLs:

# config/urls.py
from django.urls import path
from app.views import health_check

urlpatterns = [
    path('health/', health_check, name='health_check'),
    # ... other urls
]

---

Part 7: Sample Tests

# tests/test_integration.py
import pytest
from django.test import Client
from django.urls import reverse

@pytest.mark.django_db
class TestHealthCheck:
    def test_health_check_endpoint(self):
        client = Client()
        response = client.get('/health/')
        assert response.status_code == 200
        assert response.json()['status'] == 'healthy'

@pytest.mark.django_db
class TestDeploymentReadiness:
    def test_database_migrations(self):
        from django.core.management import call_command
        # This will raise an exception if there are unapplied migrations
        call_command('migrate', '--check')

    def test_static_files_collection(self):
        from django.core.management import call_command
        call_command('collectstatic', '--no-input', '--dry-run')

---

Part 8: Best Practices

8.1 Branch Protection Rules

Set up branch protection on GitHub:

• Require pull request reviews
• Require status checks to pass (all CI jobs)
• Require branches to be up to date
• Require signed commits (optional)

8.2 Environment Protection Rules

For production environment:

• Required reviewers before deployment
• Wait timer (e.g., 5 minutes)
• Deployment branches (only main branch)

8.3 Workflow Optimization Tips

1. Use job dependencies wisely - Don’t make lint wait for tests if they’re independent
2. Cache aggressively - Cache pip packages, Docker layers, node_modules
3. Fail fast - Run quick checks (linting) before expensive ones (full test suite)
4. Parallel execution - Use matrix strategy for testing multiple Python/database versions
5. Conditional steps - Skip unnecessary steps based on file changes

8.4 Monitoring and Rollback

    - name: Monitor deployment
      run: |
        # Check error rates, response times, etc.
        # If metrics exceed threshold, trigger rollback

    - name: Rollback on failure
      if: failure()
      run: |
        railway rollback

---

Part 9: Troubleshooting

Common Issues:

1. Tests fail in CI but pass locally

• Check Python versions match
• Ensure database versions match
• Verify environment variables are set
• Check for timezone issues

2. Deployment fails

• Verify secrets are configured correctly
• Check deployment platform status
• Review migration compatibility
• Ensure static files are properly collected

3. Slow pipeline

• Enable caching for dependencies
• Use Docker layer caching
• Run jobs in parallel where possible
• Use matrix strategy efficiently

---

Conclusion

You now have a production-ready CI/CD pipeline that:

• ✅ Runs comprehensive tests automatically
• ✅ Checks code quality and security
• ✅ Tests multiple Python versions
• ✅ Deploys to staging and production
• ✅ Provides health checks and monitoring
• ✅ Sends notifications on success/failure

Next Steps:

1. Add performance testing (Locust, JMeter)
2. Implement blue-green deployments
3. Add automated database backups before deployment
4. Set up comprehensive monitoring (Sentry, New Relic)
5. Implement feature flags for gradual rollouts