Script that will email the last few lines of error logs ..

September 12, 2014 6.2k views
5d0b8e2ee83fb30cf09b17a854c6d969cf5624f2
By:
sierracircle

I am working on a script that:

  • checks if mysql and apache are running
  • if not, it sends an email and then attempts to restart the non-running service

I would like to add the last few lines of the error logs to the email. Here is what I have so far. The check works, the restart works, sending email works, but am not getting my error messages from the log:

#!/bin/bash
#ver. 1 

##this script will check mysql and apache
##if that service is not running
##it will start the service and send 
##an email to you

##set your email address 
EMAIL="email@yourdomain.com"

##list your services you want to check
SERVICES=(  'mysql' 'apache2' )

#### DO NOT CHANGE anything BELOW ####

 for i in "${SERVICES[@]}"
  do
    ###IF APACHE IS NOT RUNNING####
     if [[  "$(service $i status)" =~ "not"  ]]
    then
    service $i start
    MESSAGE= "$(tail -5 /var/log/$i/error.log)"
    SUBJECT="$i down on $(hostname) $(date) "
    echo "$MESSAGE   mysql" | mail -s "$SUBJECT" "$EMAIL"
     fi
    ###IF MYSQL IS NOT RUNNING####
     if [[  "$(service $i status)" =~ "stop" ]]
    then
    service $i start
    MESSAGE= "$(tail -5 /var/log/$i/error.log)"
    SUBJECT="$i down on $(hostname) $(date) "
    echo "$MESSAGE " | mail -s "$SUBJECT" "$EMAIL"
     fi
  done

Any tips on getting info from the error logs into the email?
Also, my loop would be much better if all services had the same status message...I wonder if there is a better way of checking if the service is running that will return the same result for all services.

1 comment
  • asb MOD September 12, 2014

    Is the space after MESSAGE= just something in your question, or is that in the actual code? If I remove the space, it works for me:

    MESSAGE="$(tail -5 /var/log/$i/error.log)"
Log In to Comment
1 Answer
asb MOD September 12, 2014

For checking if the service is running or not, you could check the exit code. A well written init script should return 0 if the service is good, 1 service is dead but the /var/run pid file exists, and 3 if it is not running at all. Like:

$ service apache2 status; echo $?
 * apache2 is running
0
$ service apache2 stop
 * Stopping web server apache2
$ service apache2 status; echo $?
 * apache2 is not running
3
Reply
  • sierracircle September 12, 2014

    Thanks @adb, that worked. I was able to grab the exit code as my variable, so this script can work with any service now. I will post it here for posterity.

    This script can be used to check on mysql, apache2, or whatever service you want. This is not meant to be a SOLUTION to services that are crashing, but rather a notification and a temporary restart until you can solve the real issue.

    NOTE*: This script should be run as your root user, so it would be added to the crontab like so:

    sudo crontab -e

    and then add to the crontab like this:

    #check on services
*/1 *  * * * /your/path/to/scripts/services-script

    and here is the script.

    #!/bin/bash
#ver. 2

##this script will check whatever services 
##you want to keep an eye on
##if that service is not running
##it will (try to) start the service and send 
##an email to you

##set your email address 
EMAIL="mail@yourdomain.com"

##list your services you want to check
##you can add as many as you like
SERVICES=(  'mysql' 'apache2' )

#### DO NOT CHANGE anything BELOW ####


 for i in "${SERVICES[@]}"
  do
    ###IF SERVICE IS NOT RUNNING####

    service $i status;
    STATS=$(echo $?)
    if [[  $STATS == 3  ]]

    then
       ##TRY TO RESTART THAT SERVICE###
    service $i start

        ##IF RESTART WORKED###
        service $i status;
        RESTART=$(echo $?)
       if [[  $RESTART == 0  ]]

    then
        ##SEND AN EMAIL###
    MESSAGE="$(tail -5 /var/log/$i/error.log)"
    SUBJECT="$i down -but restarted-  on $(hostname) $(date) "
    echo "   $i  $MESSAGE "  | mail -s "$SUBJECT" "$EMAIL"
       else

        ##IF RESTART DID NOT WORK SEND A DIFFERENT EMAIL###
    MESSAGE="$(tail -5 /var/log/$i/error.log)"
    SUBJECT="$i down on $(hostname) $(date) "
    echo "   $i  $MESSAGE  . I tried to restart but it did not work"  | mail -s "$SUBJECT" "$EMAIL"

       fi
    fi

  done
  • sierracircle September 12, 2014
Have another answer? Share your knowledge.