How to set up django app + redis + celery

Hi there!

You don’t necessarily need to create a new Droplet for Redis and Celery; you can set them up on your existing Droplet as long as it has sufficient resources to handle the additional load, in some cases Redis could use quite a bit of resources like RAM.

Alternatively, you can also use a Managed Redis cluster instead of running it on a Droplet:

https://docs.digitalocean.com/products/databases/redis/

Here’s a step-by-step guide to get you started with a Redis installed on the same Droplet:

1. Installing Redis on Your Droplet

1. SSH into Your Droplet: First, connect to your DigitalOcean droplet via SSH.

2. Install Redis: Update your package lists and install Redis using the following commands:

   sudo apt-get update
   sudo apt-get install redis-server
   

3. Configure Redis (Optional): If needed, you can configure Redis by editing its configuration file:

   sudo nano /etc/redis/redis.conf
   

   After making changes, restart Redis to apply them:

   sudo systemctl restart redis.service
   

   For more details here is an in-depth guide for this as well:

   https://www.digitalocean.com/community/tutorials/how-to-install-and-secure-redis-on-ubuntu-20-04

2. Setting Up Celery in Your Django Project

1. Install Celery: Add Celery to your Django project by installing it using pip:

   pip install celery[redis]
   

2. Configure Celery: Create a new file celery.py in your Django project’s main module (the same directory as settings.py).

   from __future__ import absolute_import, unicode_literals
   import os
   from celery import Celery
   from django.conf import settings
   
   os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'your_project.settings')
   
   app = Celery('your_project')
   
   app.config_from_object('django.conf:settings', namespace='CELERY')
   app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)
   

   In your __init__.py file in the same directory, add the following to make sure the app is loaded when Django starts:

   from __future__ import absolute_import, unicode_literals
   from .celery import app as celery_app
   
   __all__ = ('celery_app',)
   

3. Update Django Settings: In your settings.py, add the Celery configuration:

   CELERY_BROKER_URL = 'redis://localhost:6379/0'
   CELERY_RESULT_BACKEND = 'redis://localhost:6379/0'
   

   Adjust the Redis URL if your Redis server is not running on the default port or if it’s running on a different machine.

3. Running Celery

1. Run Celery Worker: In your Django project directory, run the Celery worker:

   celery -A your_project worker --loglevel=info
   

2. Run Celery Beat (Optional): If you have periodic tasks, run Celery Beat:

   celery -A your_project beat --loglevel=info
   

On another note, to ensure that Redis and Celery run continuously and restart on failure or server reboot, you can set them up as system services using systemd.

1. Create a systemd Service File for Celery: Create a new file, e.g., /etc/systemd/system/celery.service, and add the following configuration (adjust paths and names as needed):

   [Unit]
   Description=Celery Service
   After=network.target
   
   [Service]
   Type=forking
   User=your_user
   Group=your_group
   WorkingDirectory=/path/to/your/django/project
   ExecStart=/path/to/your/virtualenv/bin/celery -A your_project worker --loglevel=info
   Restart=always
   
   [Install]
   WantedBy=multi-user.target
   

2. Enable and Start the Celery Service:

   sudo systemctl enable celery.service
   sudo systemctl start celery.service
   

Let me know if you have any questions!

Best,

Bobby