Related

Awastats icon and spider problem Question Question
End to end hosting services Question Question

Question

Does the monitoring agent poll for metrics, or does it look at differences in performance counters over time?

Posted July 1, 2020 214 views
Monitoring

I noticed recently that a batch job I’m running every 3h has far lower reported Disk I/O than is actually occurring (0.3 B/s rather than 30KB/s over a 20 minute window).

Most of the disk activity is concentrated in a small subset of that 20 minute window, so plausibly the monitoring agent is polling and recording the small amount of changes to file metadata and whatnot while completely missing the bulk of the data being written (as opposed to recording total changes since the last time the monitoring agent examined the disk).

Does that explanation make sense? Does the monitoring agent poll for current throughput, or is there something else going on?

Add a comment
hansvm
By:
hansvm
Add a comment

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
1 answer
bobbyiliev July 2, 2020

Hi there @hansvm,

I tried to reproduce this at my end but it seems to be working as expected.

I ran an I/O heavy operation while having iotop running and then I checked the graphs which represented the value accordingly:

Disk I/O DigitalOcean

However, if you believe that the data that you are seeing is way off the actual I/O, you could submit a GitHub issue here:

https://github.com/digitalocean/do-agent

Hope that this helps!
Regards,
Bobby

Reply Report
  • hansvm July 2, 2020

    Hi Bobby,

    Thanks for taking the time to respond.

    Perhaps I’m just misunderstanding the units involved. During a heavy I/O workload, it looks like DO is reporting 25 bytes per second for you. What did iotop report?

    FWIW, DO’s graphs have been proportional to the expected disk I/O, just low by a factor of 10,000 or so.

    Thanks again!
    Hans

    Reply Report
    • bobbyiliev July 2, 2020

      Hi there Hans,

      Hm yes indeed, good point. The value/number is correct but it was actually MB/s and not B/s.

      I will try to do some more tests to see if this is consistent. I will update you as soon as I have some more information.

      Regards,
      Bobby

      Reply Report
    • bobbyiliev July 7, 2020

      Hi there @hansvm,

      This should now be all fixed. I would suggest running apt update && apt upgrade just to make sure that you are running the latest version.

      Let me know if you have any questions.

      Regards,
      Bobby

      Reply Report
      • hansvm July 10, 2020

        Hi Bobby,

        Everything looks great! Thank you for the fast fix :)

        Do you have time to give a brief summary of or link to whatever was going on? I didn’t notice anything telling in the latest do-agent releases.

        Best Wishes,
        Hans

        Reply Report
        • bobbyiliev July 10, 2020

          Hi there @hansvm,

          Happy to hear that it looks good at your end as well!

          Actually, by the time I reported this, there was already a patch for that on the application level. As you mentioned the values were correct but were showing in B rather than MB.

          Hope that this helps!
          Regards,
          Bobby

          Reply Report
Submit an Answer

Related

Looking for something else?

Ask a new question Search for more help