Nginx + VPS newbie! Would greatly appreciate some help with caching and load optimizing!

May 10, 2014 2.3k views
I'm using a LEMP stack on a 512mb droplet. My site gathers news articles from many different RSS feeds and adds them to a MySQL database. This process happens every 15 minutes. The users can choose to view what kind of articles they'd like to see but cannot interact with the content any more than that. The site displays content from the database via PHP through a template engine. The site uses cookies to store the categories of articles the user wants to see. I've figured a full page caching solution would be a good balance between amount of work put in and and performance gained. Previously I've been caching full pages on disk and purging them every 15 minutes when the MySQL database updates. So mostly, I'm just serving static html, css, js, jpg files with a PHP file here and there. Now I'm trying to cache the webpages in RAM instead. From my tests it doesn't seem to load any faster, but I'm guessing loading from RAM than HDD would be better under load? I've tried FastCGI_cache and memcached so far. They both seem identical. Would you guys know what might be better for my situation? I noticed FastCGI_cache hits my index.php far more often than memcached. ------------------------------------------------------------------------------------------------------------- I'm trying use and to load test. Under memcached, the site starts to break down at around 650 hits/s, but I don't see the CPU/RAM usage go higher than 50%. Does this mean my server can handle more but the server environment / my code isn't optimized enough? Also these load tests are for only 30 - 60 seconds, can I expect CPU+Memory usage to go up with time with a constant rate of hits? I checked my NGINX log and found hundreds of these: 2014/05/09 22:26:38 [alert] 1347#0: *18163 socket() failed (24: Too many open files) while connecting to upstream, client: server: xxxxxxxxx, request: "GET / HTTP/1.1", upstream: After some Googling it seems like I have to raise my ulimit and tweak NGINX workers and worker_processes. Since I'm on the 512 droplet, I have them set at 1 worker and 1024 connections, like a lot of tutorials recommended. My next course of action is to slowly raise worker_connections, ulimits and keep doing load tests. It is a tedious process and I'd very much appreciate and tips or tricks you might have for me :)
1 Answer
Here are some things that I have run into when optimizing my LEP server:

1. Build a custom install of nginx without required modules and added Google's pagespeed module to it.

2. Using pagespeed directives, use Memcached to store it in memory (not always faster than using HDD - test and test).

3. Don't forget to add memcached to php-fpm's ini file.
session.save_handler = memcache
session.save_path = unix:/tmp/memcached.sock

4. Open /etc/memcached.conf and change the memory allocated -m 128

5. Enable open_file_cache

6. Enable gzip compression

Again, these are just a few of the things that I do initially. The most important being, IMO building your own nginx install.
Have another answer? Share your knowledge.