cron job to auto restart mysql or httpd or even droplet when errors

February 9, 2015 3.3k views

Hi all,

Sometimes my droplet has a problem Error establishing database connection.
Im fixing it now. The temp way is to restart mysql then things would be ok for few hours or days.

Im also thinking a temporary way until i can truly solve the above problem.
Step A. Auto restart mysql whenever error happens => I created a cron job to do this.
Step B. But sometimes even the mysql or http were hanging out and cant be restarted. In this time I had to poweroff my droplet then turn it on again.

So how to define the way for this cron job?
I knew how to do A but not B.
Mine: Wordpress, Centos 6.5 64, Virtualmin

Thank you,
PS: Im not a tech guy so pls dont use complex jargons :-)

3 Answers

The most common reason for MySQL to crash like this is that the process is running out of memory. Rather than treat the symptom by restarting the process (this will eventually lead to data corruption due to the crashes) we can treat the cause. There are a couple things you can do.

1.) Set up a swap file on your droplet. This file will be used when your physical memory is exhausted, swapping data out here rather than allowing services to crash.

2.) Optimize the processes running on your droplet. A number of tutorials on optimizing various apps can be found here.

3.) Finally, you can choose to upgrade to a droplet plan which provides more RAM.

A useful command while doing this is

free -m

This will display a summary of your used and available memory and help you to track how your changes and optimizations are affecting resource use.

by Justin Ellingwood
Swap space can be used as an "overflow" area for your system when you run out of RAM. The operating system can store data that would normally be kept in RAM on the hard drive in a specially formatted file. In this guide, we'll demonstrate how to create and use one of these files in Ubuntu 14.04.
  • I agree with ryanpq that it is most important to treat the cause of MySQL server crashes. However, in answer to your question:

    try this first when you cannot restart mysql:

    sudo /etc/init.d/mysql stop
  • Hi ryanpq,

    I already have a swap 1Gb. My current plan has 4Gb memory and usually it just used up to 25-30% of total capacity???
    For the optiimzation, did you meant sth like this "optimize centos processes" or optimize apache performance?

    Hi sierracircle,
    usually when it was restarted then it said: stop successfully, but failed to start again.. was it different with the standalone stop command?

    Thank you all,

if you have a cron job to restart apache and mysql sometimes the service can't be correctly stopped, this is some kind of issue with users, so you need to kill the process, you can try to read pid file for process id and kill -9 that id, then restart.

Again as suggested, fix MySQL problem or this will solve nothing, only cause you a db corruption.

here some samples for restarts, you can try to add a counter of tries, if you restart X service more than 3 times in a row then #reboot:

!/bin/bash Memcached

if service memcached status | grep -q "running"; then
echo "MemCached is up"
service memcached restart
cat /root/alert.txt | mail -s "MemCached Service Down" myemail@gmail.com

This if is just a sample of reboot, you need to use correct syntax here Sample end Nginx + counter

if service nginx status | grep -q "running"; then
echo "Nginx is up"
echo 0 > /root/count.txt
service nginx restart
cat /root/alert.txt | mail -s "Nginx Service Down" myemail@gmail.com
echo 1 >> /root/count.txt


if service php-fpm status | grep -q "running"; then
echo "PHP-FPM is up"
service php-fpm restart
cat /root/alert.txt | mail -s "PHP-FPM Service Down" myemail@gmail.com

PHP-FPM Max Children Status

if curl -s http://site.com/status | grep -q "max children reached: 0"; then
echo PHP running normal;
service php-fpm reload
cat /root/alert.txt | mail -s "PHP-FPM Max Children reached!" myemail@gmail.com

thanks EpicCDN for your detailed answer :-)

Have another answer? Share your knowledge.