The author selected the Apache Software Foundation to receive a donation as part of the Write for DOnations program.
Redis (Remote Dictionary Server ) is an in-memory open source software. It is a data-structure store that uses a server’s RAM, which is several times faster than even the fastest Solid State Drive (SSD). This makes Redis highly responsive, and therefore, suitable for rate limiting.
Rate limiting is a technology that puts a cap on the number of times a user can request a resource from a server. Many services implement rate limiting to prevent abuse to a service when a user may try to put too much load on a server.
For instance, when you’re implementing a public API (Application Programming Interface) for your web application with PHP, you need some form of rate limiting. The reason is that when you release an API to the public, you’d want to put a control on the number of times an application user can repeat an action in a specific timeframe. Without any control, users may bring your system to a complete halt.
Rejecting users’ requests that exceed a certain limit allows your application to run smoothly. If you have a lot of customers, rate limiting enforces a fair-usage policy that allows each customer to have high-speed access to your application. Rate limiting is also good for reducing bandwidth costs and minimizing congestion on your server.
It might be practical to code a rate-limiting module by logging user activities in a database like MySQL. However, the end product may not be scalable when many users access the system since the data must be fetched from disk and compared against the set limit. This is not only slow, but relational database management systems are not designed for this purpose.
Since Redis works as an in-memory database, it is a qualified candidate for creating a rate limiter, and it has been proven reliable for this purpose.
In this tutorial, you’ll implement a PHP script for rate limiting with Redis on an Ubuntu 20.04 server.
Before you begin, you’ll need the following:
An Ubuntu 20.04 server and a non-root user with sudo privileges. Refer to the Initial Server Setup with Ubuntu 20.04 guide to set up your server and create a new user.
A LAMP stack. Follow How To Install Linux, Apache, MySQL, PHP (LAMP) stack on Ubuntu 20.04. In this guide, you can skip Step 4 — Creating a Virtual Host for your Website and use the default virtual host already created with Apache’s installation.
A Redis Server. Set this up by following the How To Install and Secure Redis on Ubuntu 20.04 - Quickstart tutorial.
First, you’ll begin by updating your Ubuntu server package repository index. Then, install the php-redis
extension. This is a library that allows you to implement Redis in your PHP code. To do this, run the following commands:
- sudo apt update
- sudo apt install -y php-redis
Next, restart the Apache server to load the php-redis
library:
- sudo systemctl restart apache2
Once you’ve updated your software information index and installed the Redis library for PHP, you’ll now create a PHP resource that caps users’ access based on their IP address.
In this step, you’ll create a test.php
file in the root directory (/var/www/html/
) of your web server. This file will be accessible to the public and users can type its address in a web browser to run it. However, for the basis of this guide, you’ll later test access to the resource using the curl
command.
The sample resource file allows users to access it three times in a timeframe of 10 seconds. Users trying to exceed the limit will get an error informing them that they have been rate limited.
The core functionality of this file relies heavily on the Redis server. When a user requests the resource for the first time, the PHP code in the file will create a key on the Redis server based on the user’s IP address.
When the user visits the resource again, the PHP code will try to match the user’s IP address with the keys stored in the Redis server and increment the value by one if the key exists. The PHP code will keep checking if the incremented value hits the maximum limit set.
The Redis key, which is based on the user’s IP address, will expire after 10 seconds; after this time period, logging the user’s visits to the web resource will begin again.
To begin, open the /var/www/html/test.php
file:
- sudo nano /var/www/html/test.php
Next, enter the following information to initialize the Redis class. Remember to enter the appropriate value for REDIS_PASSWORD
:
<?php
$redis = new Redis();
$redis->connect('127.0.0.1', 6379);
$redis->auth('REDIS_PASSWORD');
$redis->auth
implements plain text authentication to the Redis server. This is OK while you’re working locally (via localhost
), but if you’re using a remote Redis server, consider using SSL authentication.
Next, in the same file, initialize the following variables:
. . .
$max_calls_limit = 3;
$time_period = 10;
$total_user_calls = 0;
You’ve defined:
$max_calls_limit
: is the maximum number of calls a user can access the resource.$time_period
: defines the timeframe in seconds within which a user is allowed to access the resource per the $max_calls_limit
.$total_user_calls
: initializes a variable that retrieves the number of times a user has requested access to the resource in the given timeframe.Next, add the following code to retrieve the IP address of the user requesting the web resource:
. . .
if (!empty($_SERVER['HTTP_CLIENT_IP'])) {
$user_ip_address = $_SERVER['HTTP_CLIENT_IP'];
} elseif (!empty($_SERVER['HTTP_X_FORWARDED_FOR'])) {
$user_ip_address = $_SERVER['HTTP_X_FORWARDED_FOR'];
} else {
$user_ip_address = $_SERVER['REMOTE_ADDR'];
}
While this code uses the users’ IP address for demonstration purposes, if you’ve got a protected resource on the server that requires authentication, you might log users’ activities using their usernames or access tokens.
In such a scenario, every user authenticated into your system will have a unique identifier (for example, a customer ID, developer ID, vendor ID, or even a user ID). (If you configure this, remember to use these identifiers in place of the $user_ip_address
.)
For this guide, the user IP address is sufficient for proving the concept. So, once you’ve retrieved the user’s IP address in the previous code snippet, add the next code block to your file:
. . .
if (!$redis->exists($user_ip_address)) {
$redis->set($user_ip_address, 1);
$redis->expire($user_ip_address, $time_period);
$total_user_calls = 1;
} else {
$redis->INCR($user_ip_address);
$total_user_calls = $redis->get($user_ip_address);
if ($total_user_calls > $max_calls_limit) {
echo "User " . $user_ip_address . " limit exceeded.";
exit();
}
}
echo "Welcome " . $user_ip_address . " total calls made " . $total_user_calls . " in " . $time_period . " seconds";
In this code, you use an if...else
statement to check if there is a key defined with the IP address on the Redis server. If the key doesn’t exist, if (!$redis->exists($user_ip_address)) {...}
, you set it and define its value to 1
using the code $redis->set($user_ip_address, 1);
.
The $redis->expire($user_ip_address, $time_period);
sets the key to expire within the time period—in this case, 10
seconds.
If the user’s IP address does not exist as a Redis key, you set the variable $total_user_calls
to 1
.
In the ...else {...}...
statement block, you use the $redis->INCR($user_ip_address);
command to increment the value of the Redis key set for each IP address key by 1
. This only happens when the key is already set in the Redis server and counts as a repeat request.
The statement $total_user_calls = $redis->get($user_ip_address);
retrieves the total requests the user makes by checking their IP address-based key on the Redis server.
Toward the end of the file, you use the ...if ($total_user_calls > $max_calls_limit) {... }..
statement to check if the limit is exceeded; if so, you alert the user with echo "User " . $user_ip_address . " limit exceeded.";
. Finally, you’re informing the user about the visits they make in the time period using the echo "Welcome " . $user_ip_address . " total calls made " . $total_user_calls . " in " . $time_period . " seconds";
statement.
After adding all the code, your /var/www/html/test.php
file will be as follows:
<?php
$redis = new Redis();
$redis->connect('127.0.0.1', 6379);
$redis->auth('REDIS_PASSWORD');
$max_calls_limit = 3;
$time_period = 10;
$total_user_calls = 0;
if (!empty($_SERVER['HTTP_CLIENT_IP'])) {
$user_ip_address = $_SERVER['HTTP_CLIENT_IP'];
} elseif (!empty($_SERVER['HTTP_X_FORWARDED_FOR'])) {
$user_ip_address = $_SERVER['HTTP_X_FORWARDED_FOR'];
} else {
$user_ip_address = $_SERVER['REMOTE_ADDR'];
}
if (!$redis->exists($user_ip_address)) {
$redis->set($user_ip_address, 1);
$redis->expire($user_ip_address, $time_period);
$total_user_calls = 1;
} else {
$redis->INCR($user_ip_address);
$total_user_calls = $redis->get($user_ip_address);
if ($total_user_calls > $max_calls_limit) {
echo "User " . $user_ip_address . " limit exceeded.";
exit();
}
}
echo "Welcome " . $user_ip_address . " total calls made " . $total_user_calls . " in " . $time_period . " seconds";
When you’ve finished editing the /var/www/html/test.php
file, save and close it.
You’ve now coded the logic needed to rate limit users on the test.php
web resource. In the next step, you’ll test your script.
In this step, you’ll use the curl
command to request the web resource that you’ve coded in Step 2. To fully check the script, you’ll request the resource five times in a single command. It is possible to do this by including a placeholder URL parameter at the end of the test.php
file. Here, you use the value ?[1-5]
at the end of your request to execute the curl
commands five times.
Run the following command:
- curl -H "Accept: text/plain" -H "Content-Type: text/plain" -X GET http://localhost/test.php?[1-5]
After running the code, you will receive output similar to the following:
Output[1/5]: http://localhost/test.php?1 --> <stdout>
--_curl_--http://localhost/test.php?1
Welcome 127.0.0.1 total calls made 1 in 10 seconds
[2/5]: http://localhost/test.php?2 --> <stdout>
--_curl_--http://localhost/test.php?2
Welcome 127.0.0.1 total calls made 2 in 10 seconds
[3/5]: http://localhost/test.php?3 --> <stdout>
--_curl_--http://localhost/test.php?3
Welcome 127.0.0.1 total calls made 3 in 10 seconds
[4/5]: http://localhost/test.php?4 --> <stdout>
--_curl_--http://localhost/test.php?4
User 127.0.0.1 limit exceeded.
[5/5]: http://localhost/test.php?5 --> <stdout>
--_curl_--http://localhost/test.php?5
User 127.0.0.1 limit exceeded.
As you’ll note, the first three requests ran without a problem. However, your script has rate limited the fourth and fifth requests. This confirms that the Redis server is rate limiting users’ requests.
In this guide, you’ve set low values for the two variables following:
...
$max_calls_limit = 3;
$time_period = 10;
...
When designing your application in a production environment, you could consider higher values depending on how often you expect users to hit your application.
It is best practice to check real-time stats before setting these values. For instance, if your server logs show that an average user hits your application 1,000 times every 60 seconds, you may use those values as a benchmark for throttling users.
To put things in a better perspective, here are some real-world examples of rate-limiting implementations (as of 2021):
/statuses/mentions_timeline
and /statuses/user_timeline
endpoints.This tutorial implemented a PHP script for rate limiting with Redis on an Ubuntu 20.04 server to prevent your web application from inadvertent or malicious overuse. You could extend the code to further suit your needs depending on your use case.
You might want to secure your Apache server for production use; follow the How To Secure Apache with Let’s Encrypt on Ubuntu 20.04 tutorial.
You might also consider reading how Redis works as a database cache. Try out our How To Set Up Redis as a Cache for MySQL with PHP on Ubuntu 20.04 tutorial.
You can find further resources on our PHP and Redis topic pages.
Thanks for learning with the DigitalOcean Community. Check out our offerings for compute, storage, networking, and managed databases.
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
Sign up for Infrastructure as a Newsletter.
Working on improving health and education, reducing inequality, and spurring economic growth? We'd like to help.
Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.
This comment has been deleted