semipro
By:
semipro

Error Establishing Database Connection

July 21, 2017 199 views
WordPress CentOS

I've been having this issue since I setup this droplet roughly a month ago. I'm paying for the $10 WordPress server which should be more than capable of handing this website. It's 10 pages are receiving virtually no traffic. I have websites on the $5 servers that are much larger and handle daily traffic fine.

I've installed htop and sometimes if I click around too quickly in Wordpress it causes the server to use to much ram. That's not much of an issue. The issue is the server randomly crashing hundreds of times a day.

I don't believe it's my theme or my plugins. I've used this same theme and plugins on other websites with no problems.

Also, I followed the instruction on how to turn off XML-RPC so that won't be the cause.

I have hundreds of these each day.
/var/log/mysql/error.log:2017-07-21T14:16:46.127753Z 0 [ERROR] InnoDB: Cannot allocate memory for the buffer pool
/var/log/mysql/error.log:2017-07-21T14:17:16.670788Z 0 [ERROR] InnoDB: Cannot allocate memory for the buffer pool
/var/log/mysql/error.log:2017-07-21T14:17:47.151463Z 0 [ERROR] InnoDB: Cannot allocate memory for the buffer pool
/var/log/mysql/error.log:2017-07-21T14:18:17.705673Z 0 [ERROR] InnoDB: Cannot allocate memory for the buffer pool

2017-07-21T20:33:11.925586Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is ```
deprecated. Please use --explicitdefaultsfortimestamp server option (see documentation for more details).
2017-07-21T20:33:11.930052Z 0 [Note] /usr/sbin/mysqld (mysqld 5.7.19-0ubuntu0.16.04.1) starting as process 14555 ...
2017-07-21T20:33:11.949845Z 0 [Note] InnoDB: PUNCH HOLE support available
2017-07-21T20:33:11.949895Z 0 [Note] InnoDB: Mutexes and rw
locks use GCC atomic builtins
2017-07-21T20:33:11.949905Z 0 [Note] InnoDB: Uses event mutexes
2017-07-21T20:33:11.949911Z 0 [Note] InnoDB: GCC builtin _atomicthreadfence() is used for memory barrier
2017-07-21T20:33:11.949916Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.8
2017-07-21T20:33:11.949937Z 0 [Note] InnoDB: Using Linux native AIO
2017-07-21T20:33:11.951436Z 0 [Note] InnoDB: Number of pools: 1
2017-07-21T20:33:11.953206Z 0 [Note] InnoDB: Using CPU crc32 instructions
2017-07-21T20:33:11.957165Z 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2017-07-21T20:33:11.976546Z 0 [Note] InnoDB: Completed initialization of buffer pool
2017-07-21T20:33:11.982923Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2017-07-21T20:33:12.003975Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
2017-07-21T20:33:12.008146Z 0 [Note] InnoDB: Log scan progressed past the checkpoint lsn 262544310
2017-07-21T20:33:12.008204Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 262544319
2017-07-21T20:33:12.008216Z 0 [Note] InnoDB: Database was not shutdown normally!
2017-07-21T20:33:12.008222Z 0 [Note] InnoDB: Starting crash recovery.
2017-07-21T20:33:12.208095Z 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2017-07-21T20:33:12.208142Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2017-07-21T20:33:12.208229Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2017-07-21T20:33:12.253431Z 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2017-07-21T20:33:12.254339Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active.
2017-07-21T20:33:12.254352Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
2017-07-21T20:33:12.256557Z 0 [Note] InnoDB: Waiting for purge to start
2017-07-21T20:33:12.306871Z 0 [Note] InnoDB: 5.7.19 started; log sequence number 262544319
2017-07-21T20:33:12.309148Z 0 [Note] Plugin 'FEDERATED' is disabled.
2017-07-21T20:33:12.309557Z 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib
buffer_pool
2017-07-21T20:33:12.334812Z 0 [Note] InnoDB: Buffer pool(s) load completed at 170721 20:33:12
2017-07-21T20:33:12.336702Z 0 [Warning] Failed to set up SSL because of the following SSL library error: SSL context is not usable without certificate and private key
2017-07-21T20:33:12.336733Z 0 [Note] Server hostname (bind-address): '127.0.0.1'; port: 3306
2017-07-21T20:33:12.336749Z 0 [Note] - '127.0.0.1' resolves to '127.0.0.1';
2017-07-21T20:33:12.336810Z 0 [Note] Server socket created on IP: '127.0.0.1'.
2017-07-21T20:33:12.371365Z 0 [Note] Event Scheduler: Loaded 0 events
2017-07-21T20:33:12.371654Z 0 [Note] /usr/sbin/mysqld: ready for connections.
engine. You may use the startup option '--disable-partition-engine-check' to skip this check.
2017-07-21T20:33:12.371671Z 0 [Note] Beginning of list of non-natively partitioned tables
2017-07-21T20:33:12.423844Z 0 [Note] End of list of non-natively partitioned tables
2017-07-21T20:33:12.674749Z 3 [Note] Access denied for user 'root'@'localhost' (using password: NO)

3 Answers
semipro July 23, 2017
Accepted Answer

Thank you, everyone, for the advice and suggestions. I think I've figured out the issue.
I'm using CloudFlare CDN and SSL. This is a popular service that many people use to make their websites faster and more secure. One of the features of CloudFlare is to take snapshots of the website. If your website is ever offline, CloudFlare will serve up this snapshot. Unfortunately, CloudFlare was taking these snapshots multiple times per hour, and loading what appears to be my entire website all at the same time. This was causing the overuse of ram. Since I've turned that feature off I haven't had a single crash. Hopefully, this will help other people having similar issues.

Fingers crossed that it's fixed.

  • Cool i have a Cloudflare free account active on 2 of my sites, how i can disable CDN and SSL?

Hi @semipro

Can you monitor your access-log to see if your server might be attacked with massive amounts of requests.
You're only giving about 200MB to MySQL - and CentOS is probably taking another 200MB, which should give you 600MB for Apache+PHP, so you're right, that should be enough for a typical WordPress site with low traffic.
It doesn't look like your problem is MySQL, but probably Apache+PHP - I'm not a big fan of Apache and it's mod_php module, so my recommendation would be Nginx with PHP-FPM.

  • Sorry, I don't know much about operating these servers. How would I go about doing that. Also, restarting mysql is one of my go to ways of getting the site back up. So it may be mysql related.

    • @semipro

      Just because MySQL is crashing, doesn't necessarily it's the root cause.
      MySQL is being crashed because it's the biggest consumer of RAM in a single process.
      Apache runs with many processes, so in total it might eat more RAM, but the single thread doesn't.
      If you had a $5 droplet, then I would say the problem is MySQL, since you don't have enough RAM left to run the database, but since you said it's a $10 droplet, then it should be enough for light traffic on a single WordPress site.

      There has been multiple people seeing high number of attacks lately on their WordPress installations - not only on the XML-RPC - so using something like Fail2ban would help mitigate such attacks.
      https://www.digitalocean.com/community/tutorials/how-to-protect-ssh-with-fail2ban-on-centos-7
      https://www.digitalocean.com/community/tutorials/how-to-protect-wordpress-with-fail2ban-on-ubuntu-14-04

      Your access log is located in /var/log/apache2/access.log unless you've modified the default location.
      And you can run an observer on that file, which will continue to display the updated log until closed:

      tail -f /var/log/apache2/access.log
      # Press CTRL+C to exit
      

      You might want to look at ServerPilot or something similar, if you're not familiar with keeping the server in-tune.

      WordPress is a very robust content-management system (CMS) that is free and open source. Because anyone can comment, create an account, and post on WordPress, many malicious actors have created networks of bots and servers that compromise and spam WordPress sites through brute-force attacks. The tool Fail2ban is useful in preventing unauthorized access to both your Droplet and your WordPress site. It notes suspicious or repeated login failures and proactively bans those IPs by modifying firewall
      • Okay, I'm monitoring the access log. It doesn't appear that I'm under attack, at least not currently. I'll keep monitoring it.

        • @semipro

          Every time you click around on your website, the log should update - if not, it's not the correct log file.

          Have you updated CentOS, Apache, PHP, MySQL? If it's older versions, then there might be memory bugs, which could be the issue.

          • Yeah, that's exactly what's happening.

            The only updating I've done is sudo apt-get update and upgrade

Have you tried to add swap ? Add some swap and see if fixes your problem

You have here a tutorial on how to add swap:
https://www.digitalocean.com/community/tutorials/how-to-add-swap-on-centos-6

by Etel Sverdlov
Linux swaps allow a system to harness more memory than was originally physically available. Here's how to set up a linux swap file on CentOS 6
Have another answer? Share your knowledge.