How To Implement PHP Rate Limiting with Redis on Ubuntu 20.04

The author selected the Apache Software Foundation to receive a donation as part of the Write for DOnations program.

Introduction

Redis (Remote Dictionary Server ) is an in-memory open source software. It is a data-structure store that uses a server’s RAM, which is several times faster than even the fastest Solid State Drive (SSD). This makes Redis highly responsive, and therefore, suitable for rate limiting.

Rate limiting is a technology that puts a cap on the number of times a user can request a resource from a server. Many services implement rate limiting to prevent abuse to a service when a user may try to put too much load on a server.

For instance, when you’re implementing a public API (Application Programming Interface) for your web application with PHP, you need some form of rate limiting. The reason is that when you release an API to the public, you’d want to put a control on the number of times an application user can repeat an action in a specific timeframe. Without any control, users may bring your system to a complete halt.

Rejecting users’ requests that exceed a certain limit allows your application to run smoothly. If you have a lot of customers, rate limiting enforces a fair-usage policy that allows each customer to have high-speed access to your application. Rate limiting is also good for reducing bandwidth costs and minimizing congestion on your server.

It might be practical to code a rate-limiting module by logging user activities in a database like MySQL. However, the end product may not be scalable when many users access the system since the data must be fetched from disk and compared against the set limit. This is not only slow, but relational database management systems are not designed for this purpose.

Since Redis works as an in-memory database, it is a qualified candidate for creating a rate limiter, and it has been proven reliable for this purpose.

In this tutorial, you’ll implement a PHP script for rate limiting with Redis on an Ubuntu 20.04 server.

Prerequisites

Before you begin, you’ll need the following:

• An Ubuntu 20.04 server and a non-root user with sudo privileges. Refer to the Initial Server Setup with Ubuntu 20.04 guide to set up your server and create a new user.

• A LAMP stack. Follow How To Install Linux, Apache, MySQL, PHP (LAMP) stack on Ubuntu 20.04. In this guide, you can skip Step 4 — Creating a Virtual Host for your Website and use the default virtual host already created with Apache’s installation.

• A Redis Server. Set this up by following the How To Install and Secure Redis on Ubuntu 20.04 - Quickstart tutorial.

Step 1 — Installing the Redis Library for PHP

First, you’ll begin by updating your Ubuntu server package repository index. Then, install the php-redis extension. This is a library that allows you to implement Redis in your PHP code. To do this, run the following commands:

sudo apt update
sudo apt install -y php-redis

Next, restart the Apache server to load the php-redis library:

sudo systemctl restart apache2

Once you’ve updated your software information index and installed the Redis library for PHP, you’ll now create a PHP resource that caps users’ access based on their IP address.

Step 2 — Building a PHP Web Resource for Rate Limiting

In this step, you’ll create a test.php file in the root directory (/var/www/html/) of your web server. This file will be accessible to the public and users can type its address in a web browser to run it. However, for the basis of this guide, you’ll later test access to the resource using the curl command.

The sample resource file allows users to access it three times in a timeframe of 10 seconds. Users trying to exceed the limit will get an error informing them that they have been rate limited.

The core functionality of this file relies heavily on the Redis server. When a user requests the resource for the first time, the PHP code in the file will create a key on the Redis server based on the user’s IP address.

When the user visits the resource again, the PHP code will try to match the user’s IP address with the keys stored in the Redis server and increment the value by one if the key exists. The PHP code will keep checking if the incremented value hits the maximum limit set.

The Redis key, which is based on the user’s IP address, will expire after 10 seconds; after this time period, logging the user’s visits to the web resource will begin again.

To begin, open the /var/www/html/test.php file:

sudo nano /var/www/html/test.php

Next, enter the following information to initialize the Redis class. Remember to enter the appropriate value for REDIS_PASSWORD:

<?php

$redis = new Redis();
$redis->connect('127.0.0.1', 6379);
$redis->auth('REDIS_PASSWORD');

$redis->auth implements plain text authentication to the Redis server. This is OK while you’re working locally (via localhost), but if you’re using a remote Redis server, consider using SSL authentication.

Next, in the same file, initialize the following variables:

. . .
$max_calls_limit  = 3;
$time_period      = 10;
$total_user_calls = 0;

You’ve defined:

• $max_calls_limit: is the maximum number of calls a user can access the resource.
• $time_period: defines the timeframe in seconds within which a user is allowed to access the resource per the $max_calls_limit.
• $total_user_calls: initializes a variable that retrieves the number of times a user has requested access to the resource in the given timeframe.

Next, add the following code to retrieve the IP address of the user requesting the web resource:

. . .
if (!empty($_SERVER['HTTP_CLIENT_IP'])) {
    $user_ip_address = $_SERVER['HTTP_CLIENT_IP'];
} elseif (!empty($_SERVER['HTTP_X_FORWARDED_FOR'])) {
    $user_ip_address = $_SERVER['HTTP_X_FORWARDED_FOR'];
} else {
    $user_ip_address = $_SERVER['REMOTE_ADDR'];
}

While this code uses the users’ IP address for demonstration purposes, if you’ve got a protected resource on the server that requires authentication, you might log users’ activities using their usernames or access tokens.

In such a scenario, every user authenticated into your system will have a unique identifier (for example, a customer ID, developer ID, vendor ID, or even a user ID). (If you configure this, remember to use these identifiers in place of the $user_ip_address.)

For this guide, the user IP address is sufficient for proving the concept. So, once you’ve retrieved the user’s IP address in the previous code snippet, add the next code block to your file:

. . .
if (!$redis->exists($user_ip_address)) {
    $redis->set($user_ip_address, 1);
    $redis->expire($user_ip_address, $time_period);
    $total_user_calls = 1;
} else {
    $redis->INCR($user_ip_address);
    $total_user_calls = $redis->get($user_ip_address);
    if ($total_user_calls > $max_calls_limit) {
        echo "User " . $user_ip_address . " limit exceeded.";
        exit();
    }
}

echo "Welcome " . $user_ip_address . " total calls made " . $total_user_calls . " in " . $time_period . " seconds";

In this code, you use an if...else statement to check if there is a key defined with the IP address on the Redis server. If the key doesn’t exist, if (!$redis->exists($user_ip_address)) {...}, you set it and define its value to 1 using the code $redis->set($user_ip_address, 1);.

The $redis->expire($user_ip_address, $time_period); sets the key to expire within the time period—in this case, 10 seconds.

If the user’s IP address does not exist as a Redis key, you set the variable $total_user_calls to 1.

In the ...else {...}... statement block, you use the $redis->INCR($user_ip_address); command to increment the value of the Redis key set for each IP address key by 1. This only happens when the key is already set in the Redis server and counts as a repeat request.

The statement $total_user_calls = $redis->get($user_ip_address); retrieves the total requests the user makes by checking their IP address-based key on the Redis server.

Toward the end of the file, you use the ...if ($total_user_calls > $max_calls_limit) {... }.. statement to check if the limit is exceeded; if so, you alert the user with echo "User " . $user_ip_address . " limit exceeded.";. Finally, you’re informing the user about the visits they make in the time period using the echo "Welcome " . $user_ip_address . " total calls made " . $total_user_calls . " in " . $time_period . " seconds"; statement.

After adding all the code, your /var/www/html/test.php file will be as follows:

<?php
$redis = new Redis();
$redis->connect('127.0.0.1', 6379);
$redis->auth('REDIS_PASSWORD');

$max_calls_limit  = 3;
$time_period      = 10;
$total_user_calls = 0;

if (!empty($_SERVER['HTTP_CLIENT_IP'])) {
    $user_ip_address = $_SERVER['HTTP_CLIENT_IP'];
} elseif (!empty($_SERVER['HTTP_X_FORWARDED_FOR'])) {
    $user_ip_address = $_SERVER['HTTP_X_FORWARDED_FOR'];
} else {
    $user_ip_address = $_SERVER['REMOTE_ADDR'];
}

if (!$redis->exists($user_ip_address)) {
    $redis->set($user_ip_address, 1);
    $redis->expire($user_ip_address, $time_period);
    $total_user_calls = 1;
} else {
    $redis->INCR($user_ip_address);
    $total_user_calls = $redis->get($user_ip_address);
    if ($total_user_calls > $max_calls_limit) {
        echo "User " . $user_ip_address . " limit exceeded.";
        exit();
    }
}

echo "Welcome " . $user_ip_address . " total calls made " . $total_user_calls . " in " . $time_period . " seconds";

When you’ve finished editing the /var/www/html/test.php file, save and close it.

You’ve now coded the logic needed to rate limit users on the test.php web resource. In the next step, you’ll test your script.

Step 3 — Testing Redis Rate Limiting

In this step, you’ll use the curl command to request the web resource that you’ve coded in Step 2. To fully check the script, you’ll request the resource five times in a single command. It is possible to do this by including a placeholder URL parameter at the end of the test.php file. Here, you use the value ?[1-5] at the end of your request to execute the curl commands five times.

Run the following command:

 curl -H "Accept: text/plain" -H "Content-Type: text/plain" -X GET http://localhost/test.php?[1-5]

After running the code, you will receive output similar to the following:

Output
[1/5]: http://localhost/test.php?1 --> <stdout>
--_curl_--http://localhost/test.php?1
Welcome 127.0.0.1 total calls made 1 in 10 seconds
[2/5]: http://localhost/test.php?2 --> <stdout>
--_curl_--http://localhost/test.php?2
Welcome 127.0.0.1 total calls made 2 in 10 seconds
[3/5]: http://localhost/test.php?3 --> <stdout>
--_curl_--http://localhost/test.php?3
Welcome 127.0.0.1 total calls made 3 in 10 seconds
[4/5]: http://localhost/test.php?4 --> <stdout>
--_curl_--http://localhost/test.php?4
User 127.0.0.1 limit exceeded.
[5/5]: http://localhost/test.php?5 --> <stdout>
--_curl_--http://localhost/test.php?5
User 127.0.0.1 limit exceeded.

As you’ll note, the first three requests ran without a problem. However, your script has rate limited the fourth and fifth requests. This confirms that the Redis server is rate limiting users’ requests.

In this guide, you’ve set low values for the two variables following:

...
$max_calls_limit  = 3;
$time_period      = 10;
...

When designing your application in a production environment, you could consider higher values depending on how often you expect users to hit your application.

It is best practice to check real-time stats before setting these values. For instance, if your server logs show that an average user hits your application 1,000 times every 60 seconds, you may use those values as a benchmark for throttling users.

To put things in a better perspective, here are some real-world examples of rate-limiting implementations (as of 2021):

• Twitter allows only 100,000 requests per day to their /statuses/mentions_timeline and /statuses/user_timeline endpoints.
• DigitalOcean allows 5,000 requests per hour in their API endpoint for each user authenticated with an OAuth token.
• The Google Custom Search JSON API allows 100 search queries per day for free.

Conclusion

This tutorial implemented a PHP script for rate limiting with Redis on an Ubuntu 20.04 server to prevent your web application from inadvertent or malicious overuse. You could extend the code to further suit your needs depending on your use case.

You might want to secure your Apache server for production use; follow the How To Secure Apache with Let’s Encrypt on Ubuntu 20.04 tutorial.

You might also consider reading how Redis works as a database cache. Try out our How To Set Up Redis as a Cache for MySQL with PHP on Ubuntu 20.04 tutorial.

You can find further resources on our PHP and Redis topic pages.