Question

Droplet experiencing high load averages during low CPU usage

Posted January 1, 2020 386 views
Miscellaneous

I have a Ubuntu 18.04 droplet that occasionally experiences high load averages for brief periods, but the DO monitoring charts show low and steady CPU usage, steady disk I/O, steady traffic, and a small bump in RAM (probably due to extra apache processes waiting to run). So, everything looks normal except for the load averages. If the load averages go up into the 20s then I can see significant impact on page load times.

These episodes are not associated with high traffic, and they don’t normally last for more than a few minutes so it’s hard for me to catch them as they’re occurring. However, the one time I managed to do so I used the “top” command and saw extra apache processes (as expected during high load), but nothing else looked unusual. Nothing suspicious appears in syslog.

Any recommendations on what I need to do to troubleshoot this problem?

Add a comment
Bcbdbe0eab0de5765ae037e6ae25d6dbf895b9b8
By:
JigsawBob
Add a comment

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
1 answer
alexdo January 2, 2020

Hello, @businessbcbdbe0eab0de5765a

What you can do is to check the Apache error log and see if the Apache workers have been exceeded as this will explain the spikes in the memory usage. You can check this with the following command:

grep -i workers $APACHE_LOG_DIR (usually it’s /var/log/apache2/error.log)

If the apache workers have been exceeded then you can tweak the value in the apache configuration in order to sort the issue. The exact value you need to look for is MaxRequestWorkers

I will also recommend you to check on this tutorial as you may find it useful:

https://www.digitalocean.com/community/tutorials/how-to-monitor-cpu-use-on-digitalocean-droplets

Hope this helps.

Regards,
Alex

How To Monitor CPU Use on DigitalOcean Droplets
by Melissa Anderson
The amount of memory, the size of the cache, the speed of the input and output, and the speed and availability of processing power are key elements that affect the performance of your infrastructure. In this article, we'll focus on understanding and monitoring CPU usage.We'll describe how to use two common utilities in order to learn about your CPU usage, how to set up DigitalOcean monitoring to watch performance over time, and how to set set Alert Policies to notify you about significant chang
Reply Report
  • JigsawBob January 3, 2020

    Thank you for your response, Alex!

    There was another high load episode this morning. It was brief and the load averages were not terribly high this time, but high enough to demonstrate the problem. Here are the DO monitoring charts showing the incident:

    https://www.dropbox.com/s/yeq6fmtlb2ngdq7/High-load-spike.png?dl=0

    CPU usage shows a small bump this time, but still mostly idle. No change in disk I/O and no significant change in network traffic during the load spike.

    The apache error logs showed nothing unusual during that period, a couple of 404s. Syslog showed a couple of “UFW BLOCK” entries and a “php/sessionClean” cron entry.

    My apache2.conf file does not have a MaxRequestWorkers setting, but I’ve read the default is 256 which is higher than the total number of all processes I saw running during an earlier episode.

    It seems strange to have a high load average while the CPU is so idle. The only cases I know of where this can happen is if processes are performing uninterruptable waits on disk I/O or the on network, but there is no unusual disk I/O or network traffic in this case.

    Reply Report
    • alexdo January 7, 2020

      Hello, @businessbcbdbe0eab0de5765a

      It is indeed strange to have the load spikes when the CPU usage is mostly idle and there is no change in the disk I/O and no network traffic during this specific time.

      If you’re unable to see anything obvious that will cause the load spikes I will recommend to contact our support team as they will be able to further assist you.

      Regards,
      Alex

      Reply Report
      • JigsawBob January 7, 2020

        I think the problem was a cron task that was running every 10 minutes. The cron task was a PHP script being invoked via cURL. Apparently, cURL does not release the CPU while waiting for a response. So, one of my two CPUs was tied up by cron while the other was executing the PHP script, leaving no CPUs to handle normal traffic. It’s normally a very short running script, so I’m still not sure exactly what went wrong, but I haven’t seen any significant load spikes since disabling that cron task.

        So lesson learned… Don’t use cURL to execute a cron task!

        Reply Report
        • alexdo January 9, 2020

          Hello, @businessbcbdbe0eab0de5765a

          Thanks for explaining this, because other users may have the same issue in the future as well and this may help them to solve their issues.

          You can try to redesign the cron job and see if the CPUs will still be utilized at 100%

          Regards,
          Alex

          Reply Report
Submit an Answer

Related

Looking for something else?

Ask a new question Search for more help