Tutorial

How To Scale Django: Finding the Bottleneck

Published on November 14, 2013
author

Matthew Nuzum

How To Scale Django: Finding the Bottleneck

Introduction


Django is an excellent Python based platform for building modern web apps. One of its biggest strengths is that it helps developers work faster.

You’ve built your awesome app and deployed it. Things are great, but now that you’re loading it up with larger amounts of data and you’re starting to have several people use it at the same time, it’s not as fast as you’d like.

It’s a common problem. Fortunately, we have some tools to help alleviate the problems.

First, let’s check for a few of the more obvious issues:

Use a Real Database


During local development, it’s hard to beat SQLite3. Unless you’re careful, you might be using it on your virtual server as well.

SQLite3 doesn’t scale to multiple simultaneous users like MySQL and PostgreSQL do, especially for operations that do many write operations (if you’re using sessions then you are writing to the database).

If you’re using a low-memory VPS, for example the 512MB droplet, I’d suggest using MySQL. If you have memory to spare (2GB or more), then I’d urge you to consider PostgreSQL, since it is preferred by many of the Django developers.

Disable Debug Mode


Debug mode is absolutely necessary when doing local development, but it will slow down your production server. Look at your virtual server’s settings.py and check to ensure that DEBUG is set to False. Also confirm that TEMPLATE_DEBUG is also set to False or that it is set to DEBUG.

Use the Debug Toolbar to Pin Down Performance Problems


On your local development computer, not your production server, turn on Django Debug Toolbar to locate specific problems.

You do this by installing the django-debug-toolbar module and then adding an entry to your MIDDLEWARE_CLASSES dictionary like this:

MIDDLEWARE_CLASSES = (
    # ...
    'debug_toolbar.middleware.DebugToolbarMiddleware',
    # ...
)

You’ll also need to create an INTERNAL_IPS variable and add your IP address. If you’re developing locally, your IP address is probably 127.0.0.1, so you’d add a line like this to your settings.py:

INTERNAL_IPS = ('127.0.0.1',)

Finally, add debug_toolbar as the last item in your INSTALLED_APPS, like this:

INSTALLED_APPS = (
    # ...
    'debug_toolbar',
)

The installation documentation contains some more detail and optional configuration options you may want to consider.

Remember, don’t push these changes to production on accident! (If you do it on purpose, that’s fine).

Now you should see a black panel appear down the side of your web pages as you browse around the site. If you like stats and numbers and all kinds of geeky details, you’ll love it. You’ll also quickly see why you don’t want this on your production server!

<a href=“Signals”><img src=“https://assets.digitalocean.com/articles/scale_django/img1.png”></a>

Having built numerous Django apps, I’ll suggest you narrow in on the SQL section of the panel, since that is commonly an area of concern.

One benefit/problem that Django has is lazy loading of related fields when queries are performed. This means that Django will prefer not to do a join. If you end up needing related fields, it will do an additional query each time a related field is needed.

If you do need related fields, this can causes n+1 number of SQL queries. For example, let’s say you are going to make a replacement version of Twitter. You create a model for tweets and each model is related to the User model. On your homepage, you list the 30 latest tweets along with the name of the user who created them. This can cause you to do at least 31 SQL queries. One query to get the list of tweets, and one query for each username lookup.

The solution to this problem is select_related. This is a very simple modification to the query that causes Django to do a join when fetching data. You should use it on any lookup where you know you’ll need related fields.

You simply modify a query like this:

Entry.objects.get(id=5)

to look like this:

Entry.objects.select_related().get(id=5)

Read the documentation for this feature and use it only when necessary.

Wrap Up


In my experience, the above issues solve many sites’ initial performance problems and are all quite easy to fix with minimal code and configuration changes.

<div class=“author”>Submitted by: <a href=“http://www.bearfruit.org/”>Matthew Nuzum</div>

Thanks for learning with the DigitalOcean Community. Check out our offerings for compute, storage, networking, and managed databases.

Learn more about our products


Tutorial Series: Scaling Django

Django is a Python-based platform for building modern web apps. As with any platform, issues can arise with your need to scale and expand your user base. In this tutorial series, we will go over several strategies and best practices for scaling your Django app in a reliable, stable way.

About the authors
Default avatar
Matthew Nuzum

author

Still looking for an answer?

Ask a questionSearch for more help

Was this helpful?
 
1 Comments


This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

I think you need to update this tutorial. It doesn’t show the step of adding to the urls.py and the debug toolbar will not work without that.

Try DigitalOcean for free

Click below to sign up and get $200 of credit to try our products over 60 days!

Sign up

Join the Tech Talk
Success! Thank you! Please check your email for further details.

Please complete your information!

Featured on Community

Get our biweekly newsletter

Sign up for Infrastructure as a Newsletter.

Hollie's Hub for Good

Working on improving health and education, reducing inequality, and spurring economic growth? We'd like to help.

Become a contributor

Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.

Welcome to the developer cloud

DigitalOcean makes it simple to launch in the cloud and scale up as you grow — whether you're running one virtual machine or ten thousand.

Learn more