// Tutorial //

How To Host Multiple Websites Securely With Nginx And Php-fpm On Ubuntu 14.04

Published on July 1, 2015
Default avatar
By Toli
Developer and author at DigitalOcean.
How To Host Multiple Websites Securely With Nginx And Php-fpm On Ubuntu 14.04

Introduction

It’s well known that the LEMP stack (Linux, nginx, MySQL, PHP) provides unmatched speed and reliability for running PHP sites. Other benefits of this popular stack such as security and isolation are less popular, though.

In this article we’ll show you the security and isolation benefits of running sites on LEMP with different Linux users. This will be done by creating different php-fpm pools for each nginx server block (site or virtual host).

Prerequisites

This guide has been tested on Ubuntu 14.04. The described installation and configuration would be similar on other OS or OS versions, but the commands and location of configuration files may vary.

It also assumes you already have nginx and php-fpm set up. If not, please follow step one and step three from the article How To Install Linux, nginx, MySQL, PHP (LEMP) stack on Ubuntu 14.04.

All the commands in this tutorial should be run as a non-root user. If root access is required for the command, it will be preceded by sudo. If you don’t already have that set up, follow this tutorial: Initial Server Setup with Ubuntu 14.04.

You will also need a fully qualified domain name (fqdn) that points to the Droplet for testing in addition to the default localhost. If you don’t have one at hand, you can use site1.example.org. Edit the /etc/hosts file with your favorite editor like this sudo vim /etc/hosts and add this line (replace site1.example.org with your fqdn if you are using it):

/etc/hosts
...
127.0.0.1 site1.example.org
... 

Reasons to Secure LEMP Additionally

Under a common LEMP setup there is only one php-fpm pool which runs all PHP scripts for all sites under the same user. This poses two major problems:

  • If a web application on one nginx server block, i.e. subdomain or separate site, gets compromised, all of the sites on this Droplet will be affected too. The attacker is able to read the configuration files, including database details, of the other sites or even alter their files.
  • If you want to give a user access to a site on your Droplet, you will be practically giving him access to all sites. For example, your developer needs to work on the staging environment. However, even with very strict file permissions you will be still giving him access to all the sites, including your main site, on the same Droplet.

The above problems are solved in php-fpm by creating a different pool which runs under a different user for each site.

Step 1 — Configuring php-fpm

If you have covered the prerequisites, then you should already have one functional website on the Droplet. Unless you have specified a custom fqdn for it, you should be able to access it under the fqdn localhost locally or by the IP of the droplet remotely.

Now we’ll create a second site (site1.example.org) with its own php-fpm pool and Linux user.

Let’s start with creating the necessary user. For best isolation, the new user should have its own group. So first create the user group site1:

  1. sudo groupadd site1

Then please create an user site1 belonging to this group:

  1. sudo useradd -g site1 site1

So far the new user site1 does not have a password and cannot log in the Droplet. If you need to provide someone with direct access to the files of this site, then you should create a password for this user with the command sudo passwd site1. With the new user/password combination a user can log in remotely by ssh or sftp. For more info and security details check the article Setup a secondary SSH/SFTP user with limited directory access.

Next, create a new php-fpm pool for site1. A php-fpm pool in its very essence is just an ordinary Linux process which runs under certain user/group and listens on a Linux socket. It could also listen on an IP:port combination too but this would require more Droplet resources, and it’s not the preferred method.

By default, in Ubuntu 14.04 every php-fpm pool should be configured in a file inside the directory /etc/php5/fpm/pool.d. Every file with the extensions .conf in this directory is automatically loaded in the php-fpm global configuration.

So for our new site let’s create a new file /etc/php5/fpm/pool.d/site1.conf. You can do this with your favorite editor like this:

  1. sudo vim /etc/php5/fpm/pool.d/site1.conf

This file should contain:

/etc/php5/fpm/pool.d/site1.conf
[site1]
user = site1
group = site1
listen = /var/run/php5-fpm-site1.sock
listen.owner = www-data
listen.group = www-data
php_admin_value[disable_functions] = exec,passthru,shell_exec,system
php_admin_flag[allow_url_fopen] = off
pm = dynamic
pm.max_children = 5
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 3
chdir = /

In the above configuration note these specific options:

  • [site1] is the name of the pool. For each pool you have to specify a unique name.
  • user and group stand for the Linux user and the group under which the new pool will be running.
  • listen should point to a unique location for each pool.
  • listen.owner and listen.group define the ownership of the listener, i.e. the socket of the new php-fpm pool. Nginx must be able to read this socket. That’s why the socket is created with the user and group under which nginx runs - www-data.
  • php_admin_value allows you to set custom php configuration values. We have used it to disable functions which can run Linux commands - exec,passthru,shell_exec,system.
  • php_admin_flag is similar to php_admin_value, but it is just a switch for boolean values, i.e. on and off. We’ll disable the PHP function allow_url_fopen which allows a PHP script to open remote files and could be used by attacker.

Note: The above php_admin_value and php_admin_flag values could be also applied globally. However, a site may need them, and that’s why by default they are not configured. The beauty of php-fpm pools is that it allows you to fine tune the security settings of each site. Furthermore, these options can be used for any other php settings, outside of the security scope, to further customize the environment of a site.

The pm options are outside of the current security topic, but you should know that they allow you to configure the performance of the pool.

The chdir option should be / which is the root of the filesystem. This shouldn’t be changed unless you use another important option chroot.

The option chroot is not included in the above configuration on purpose. It would allow you to run a pool in a jailed environment, i.e. locked inside a directory. This is great for security because you can lock the pool inside the web root of the site. However, this ultimate security will cause serious problems for any decent PHP application which relies on system binaries and applications such as Imagemagick, which will not be available. If you are further interested in this topic please read the article How To Use Firejail to Set Up a WordPress Installation in a Jailed Environment.

Once you have finished with the above configuration restart php-fpm for the new settings to take effect with the command:

  1. sudo service php5-fpm restart

Verify that the new pool is properly running by searching for its processes like this:

  1. ps aux |grep site1

If you have followed the exact instructions up to here you should see output similar to:

site1   14042  0.0  0.8 133620  4208 ?        S    14:45   0:00 php-fpm: pool site1
site1   14043  0.0  1.1 133760  5892 ?        S    14:45   0:00 php-fpm: pool site1

In red is the user under which the process or the php-fpm pool runs - site1.

In addition, we’ll disable the default php caching provided by opcache. This particular caching extension might be great for performance, but it’s not for security as we’ll see later. To disable it edit the file /etc/php5/fpm/conf.d/05-opcache.ini with super user privileges and add the line:

/etc/php5/fpm/conf.d/05-opcache.ini
opcache.enable=0

Then restart again php-fpm (sudo service php5-fpm restart) for the setting to take effect.

Step 2 — Configuring nginx

Once we have configured the php-fpm pool for our site we’ll configure the server block in nginx. For this purpose please create a new file /etc/nginx/sites-available/site1 with your favorite editor like this:

  1. sudo vim /etc/nginx/sites-available/site1

This file should contain:

/etc/nginx/sites-available/site1
server {
    listen 80;

    root /usr/share/nginx/sites/site1;
    index index.php index.html index.htm;

    server_name site1.example.org;

    location / {
        try_files $uri $uri/ =404;
    }

    location ~ \.php$ {
        try_files $uri =404;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_pass unix:/var/run/php5-fpm-site1.sock;
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include fastcgi_params;
    }
}

The above code shows a common configuration for a server block in nginx. Note the interesting highlighted parts:

  • Web root is /usr/share/nginx/sites/site1.
  • The server name uses the fqdn site1.example.org which is the one mentioned in the prerequisites of this article.
  • fastcgi_pass specifies the handler for the php files. For every site you should use a different unix socket such as /var/run/php5-fpm-site1.sock.

Create the web root directory:

  1. sudo mkdir /usr/share/nginx/sites
  2. sudo mkdir /usr/share/nginx/sites/site1

To enable the above site you have to create a symlink to it in the directory /etc/nginx/sites-enabled/. This can be done with the command:

  1. sudo ln -s /etc/nginx/sites-available/site1 /etc/nginx/sites-enabled/site1

Finally, restart nginx for the change to take effect like this:

  1. sudo service nginx restart

Step 3 — Testing

For running the tests we’ll use the well-known phpinfo function which provides detailed information about the php environment. Create a new file under the name info.php which contains only the line <?php phpinfo(); ?>. You will need this file first in the the default nginx site and its web root /usr/share/nginx/html/. For this purpose you can use an editor like this:

  1. sudo vim /usr/share/nginx/html/info.php

After that copy the file to to the web root of the other site (site1.example.org) like this:

  1. sudo cp /usr/share/nginx/html/info.php /usr/share/nginx/sites/site1/

Now you are ready to run the most basic test to verify the server user. You can perform the test with a browser or from the Droplet terminal and lynx, the command line browser. If you don’t have lynx on your Droplet yet, install it with the command sudo apt-get install lynx.

First check the info.php file from your default site. It should be accessible under localhost like this:

  1. lynx --dump http://localhost/info.php |grep 'SERVER\["USER"\]'

In the above command we filter the output with grep only for the variable SERVER["USER"] which stands for the server user. For the default site the output should show the default www-data user like this:

_SERVER["USER"]                 www-data

Similarly, next check the server user for site1.example.org:

  1. lynx --dump http://site1.example.org/info.php |grep 'SERVER\["USER"\]'

You should see this time in the output the site1 user:

_SERVER["USER"]                 site1

If you have made any custom php settings on a per php-fpm pool basis, then you can also check their corresponding values in the above manner by filtering the output that interests you.

So far, we know that our two sites run under different users, but now let’s see how to secure a connection. To demonstrate the security problem we are solving in this article, we’ll create a file with sensitive information. Usually such a file contains the connection string to the database and include the user and password details of the database user. If anyone finds out that information, the person is able to do anything with the related site.

With your favorite editor create a new file in your main site /usr/share/nginx/html/config.php. That file should contain:

/usr/share/nginx/html/config.php
<?php
$pass = 'secret';
?>

In the above file we define a variable called pass which holds the value secret. Naturally, we want to restrict the access to this file, so we’ll set its permissions to 400, which give read only access to the owner of the file.

To change the permissions to 400 run the command:

  1. sudo chmod 400 /usr/share/nginx/html/config.php

Also, our main site runs under the user www-data who should be able to read this file. Thus, change the ownership of the file to that user like this:

  1. sudo chown www-data:www-data /usr/share/nginx/html/config.php

In our example we’ll use another file called /usr/share/nginx/html/readfile.php to read the secret information and print it. This file should contain the following code:

/usr/share/nginx/html/readfile.php
<?php
include('/usr/share/nginx/html/config.php');
print($pass);
?>

Change the ownership of this file to www-data as well:

  1. sudo chown www-data:www-data /usr/share/nginx/html/readfile.php

To confirm all permissions and ownerships are correct in the web root run the command ls -l /usr/share/nginx/html/. You should see output similar to:

-r-------- 1 www-data www-data  27 Jun 19 05:35 config.php
-rw-r--r-- 1 www-data www-data  68 Jun 21 16:31 readfile.php

Now access the latter file on your default site with the command lynx --dump http://localhost/readfile.php. You should be able to see printed in the output secret which shows that the file with sensitive information is accessible within the same site, which is the expected correct behavior.

Now copy the file /usr/share/nginx/html/readfile.php to your second site, site1.example.org like this:

  1. sudo cp /usr/share/nginx/html/readfile.php /usr/share/nginx/sites/site1/

To keep the site/user relations in order, make sure that within each site the files are owned by the respective site user. Do this by changing the ownership of the newly copied file to site1 with the command:

  1. sudo chown site1:site1 /usr/share/nginx/sites/site1/readfile.php

To confirm you have set the correct permissions and ownership of the file, please list the contents of the site1 web root with the command ls -l /usr/share/nginx/sites/site1/. You should see:

-rw-r--r-- 1 site1 site1  80 Jun 21 16:44 readfile.php

Then try to access the same file from site1.example.com with the command lynx --dump http://site1.example.org/readfile.php. You will only see empty space returned. Furthermore, if you search for errors in the error log of nginx with the grep command sudo grep error /var/log/nginx/error.log you will see:

2015/06/30 15:15:13 [error] 894#0: *242 FastCGI sent in stderr: "PHP message: PHP Warning:  include(/usr/share/nginx/html/config.php): failed to open stream: Permission denied in /usr/share/nginx/sites/site1/readfile.php on line 2

Note: You would also see a similar error in the lynx output if you have display_errors set to On in php-fpm configuration file /etc/php5/fpm/php.ini.

The warning shows that a script from the site1.example.org site cannot read the sensitive file config.php from the main site. Thus, sites which run under different users cannot compromise the security of each other.

If you go back to the end of configuration part of this article, you will see that we have disabled the default caching provided by opcache. If you are curious why, try to enable again opcache by setting with super user privileges opcache.enable=1 in the file /etc/php5/fpm/conf.d/05-opcache.ini and restart php5-fpm with the command sudo service php5-fpm restart.

Amazingly, if you run again the test steps in the exactly the same order, you’ll be able to read the sensitive file regardless of its ownership and permission. This problem in opcache has been reported for a long time, but by the time of this article it has not been fixed yet.

Conclusion

From a security point of view it’s essential to use php-fpm pools with a different user for every site on the same Nginx web server. Even if it comes with a small performance penalty, the benefit of such isolation could prevent serious security breaches.

The idea described in this article is not unique, and it’s present in other similar PHP isolation technologies such as SuPHP. However, the performance of all other alternatives is much worse than that of php-fpm.


Want to learn more? Join the DigitalOcean Community!

Join our DigitalOcean community of over a million developers for free! Get help and share knowledge in our Questions & Answers section, find tutorials and tools that will help you grow as a developer and scale your project or business, and subscribe to topics of interest.

Sign up
About the authors
Default avatar
Toli

author

Developer and author at DigitalOcean.

Default avatar
Tammy Fox

editor

Developer and author at DigitalOcean.

Still looking for an answer?

Was this helpful?
10 Comments

only thing i had to do different is my server uses virtual.conf in /etc/nginx/conf.d not sites-enabled.

but im getting a 502 bad gateway error with anything php, anything html my virtual domain works as it should. but when i try to access a php file i get 502 bad gateway only on the virtual host, on my main site php is working normally.

Well, you have written a nice piece but what about having all the Nginx and PHP-FPM settings Pre-configured? Yes, Cloudways is providing it All the default settings are setup when you launch your servers by few clicks. This would let you out of the clutter of SSH commands and sudo stuff. Also all the server permissions can be setup be a click from panel.

As of May 5, 2017, it looks like the issue with opcache has been fixed (as far as I can tell): http://phpmagazine.net/2017/03/phps-long-standing-security-issue-with-opcache-leaking-sensitive-data-fixed.html

AFFECTED VERSIONS: PHP7 < 7.0.14 and PHP5 < 5.6.29. Later versions are still vulnerable by default unless opcache.validate_permission=1 is enabled.

I’m running a DO droplet through ServerPilot, and phpinfo() shows opcache.use_cwd=1, opcache.validate_permission=1. Not sure if that’s specific to the ServerPilot config, but I’ve done very little customization on this server. Hopefully it’s fixed by default at this point.

Thank you Anatoliy,

but i am missing how to configure Mailing - is it based on the virtualhost (for example sending by test@domaina.com fpr virtualHost A and test@domainb.com for VirtualHost B) or do i have to configure this at the system?

Kind regards

I followed this tutorial (with some additions for behind behind cloudflare with lets encrypt SSL) but get a 403 error on the sites. Normal 403 troubleshoot tutorials don’t apply because of having the many users all isolated to their own site and in their own php-fpm workers. So it seems to be more complex to figure out…

Thanks so much @atexun Im using NGINX as a reverse proxy for a Apache. For that I used this digitalocean tutorial but then i read here on this tutorial about the importance of spliting php-fpm pool to separate apps and enhance security.

Now i configured my reverse proxy in such a way that Apache does not know that NGINX is sitting in front at all. As far as apache is concerned it is receiving request from the internet (from the users and visitors) and not from NGINX. That way i can use NGINX to manage all the SSL certs, (it’s easier for me to issue certs using NGINX). Also i have other domains and apps on the same droplet using only NGINX and not Apache

So i created a separete php-fpm pool for my app and then went to my /etc/apache2/sites-available folder, open my domainname.com.conf file and added the appropiate configuration lines, so now the app is working using php-fpm as a different pool and user.

How to uninstall all settings, I need to solve an issue related with the Discourse? :)

Hi, first, great tutorial!

My question, is possible to achieve this with PHP-FPM listening to TCP sockets?

Thanks

hi the most critical issue is " if we use this method for share hosting , it’s not secure at all . it means if some body upload a php sheller such as C99 , he can brows all over the os and it a disaster !!! The question is " what should we do more for a shared hosting ? " Thank you .

Can we install Varnish with this setup? Or it is better to use Zend Opcache? And what is with that FastCGI caching?

I m little confused, isn’t Zend Opcache cashing mod? So if we want to change we need to disable other caching mods first?

I have read https://www.digitalocean.com/community/tutorials/how-to-install-wordpress-nginx-php-and-varnish-on-ubuntu-12-04 tutorial, but there they say to remove default site enabled in Nginx, but i have many websites enabled, so what m i supposed to do?

Or if you can send me link to a better varnish tutorial or give some tips i be very happy.

thanks P.S. this tutorial is great :)