Updated Droplet for Monitoring, now sending a lot of data to syslog

September 22, 2016 6.8k views
DigitalOcean Monitoring Ubuntu
AnExiledGod
By:
AnExiledGod

I updated my droplet for the monitoring as a thing on my dashboard suggested it, it's Monitoring Beta and now it sends a bunch of lines to my syslog every 15 seconds and it's making it difficult to find the things I actually care about.
This is what it looks like, how do I disable this?

Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 Transmitting metrics to DigitalOcean.
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 Excluding disk ram0
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 Excluding disk ram1
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 Excluding disk ram2
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 Excluding disk ram3
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 Excluding disk ram4
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 Excluding disk ram5
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 Excluding disk ram6
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 Excluding disk ram7
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 Excluding disk ram8
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 Excluding disk ram9
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 Excluding disk ram10
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 Excluding disk ram11
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 Excluding disk ram12
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 Excluding disk ram13
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 Excluding disk ram14
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 Excluding disk ram15
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 Excluding disk loop0
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 Excluding disk loop1
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 Excluding disk loop2
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 Excluding disk loop3
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 Excluding disk loop4
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 Excluding disk loop5
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 Excluding disk loop6
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 Excluding disk loop7
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 Excluding disk vda1
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 meminfo field not recognized: MemAvailable
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 meminfo field not recognized: CmaTotal
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 meminfo field not recognized: CmaFree
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 Posting metrics to: https://nyc2.sonar.digitalocean.com/v1/metrics/droplet_id/21479984
Sep 21 20:12:32 ExternalChat-512mb-nyc2-s1 do-agent[22585]: 2016/09/21 20:12:32 sleeping for 15 seconds
Log In to Comment
4 Answers
xMudrii September 22, 2016

As end-users, we really don't know what it does. This is something that appeared this night, I was looking yesterday, there was no any Monitoring BETA.
Until DigitalOcean doesn't say what is it, we can only guess.

We can uninstall, which I would NOT!! recommend.
Second option is to clear out log files from it.

  • grep -v 'do-agent' /var/log/syslog

This will write all output of syslog to your terminal.
If you want to write it into some file, you can specify it:

  • grep -v 'do-agent' /var/log/syslog > ~/syslog-noagent

This will create syslog-noagent file without do-agent logs in your home directory.
The only problem with this is that do-agent logs will still remain in original file.

To remove previous logs permanently

  • sed --in-place '/do-agent/d' /var/log/syslog

New logs will still appear but you will have clean syslog of previous do-agent(s).

Anyways, I'm gonna observe droplet CPU, IO usage and I will write back or report to support if I see something unusual.
Writing (min) 31* logs every 15 seconds in syslog is something unacceptable

Reply
  • AnExiledGod September 22, 2016

    Thank you, I can clear out do-agent logs for now so I can at least see what's going on. My mysql went down this morning and I can't even figure out if it threw anything into syslog because it's being spammed with this useless crap.

    It's frustrating that they push out a feature like this without any information, or an explanation, or anything. Sending so much data to an important log is not at all acceptable, and I hope they fix this VERY quickly.

    • xMudrii September 22, 2016

      This looks like optional, so I think for now, this could be safe to stop or even delete on some time. Let's assume that 95% droplets doesn't have it because I don't think people paid attention to this and their droplet is working perfectly, so your droplet should work normal too.
      I don't know what to do or recommend, yes or no, that is something on you but if you need clean access to logs, you can remove or stop it.

      To remove it (WARNING: read edit on end of post!)

      • sudo apt-get remove do-agent

      • sudo systemctl daemon-reload

      To temporarily disable it:

      • sudo systemctl stop do-agent

      When you are ready to enable:

      • sudo systemctl start do-agent

      I hope they will fix soon and release what does it even do :D

      Edit: I tried reinstalling it. It gives error out:

      Failed to start do-agent.service: Unit do-agent.service not found.
dpkg: error processing package do-agent (--configure):
 subprocess installed post-installation script returned error exit status 5
Errors were encountered while processing:
 do-agent
E: Sub-process /usr/bin/dpkg returned an error code (1)

      I tried both via apt-get install do-agent or via script again. Both errors. I will report it to their support, I think this is bug under script, if already installed, it will not create service, or something like that

      • eris MOD September 22, 2016

        It's a small piece of a beta feature that was mistakenly deployed to users without the beta flag set.

        If you express interest in the beta at https://beta.digitalocean.com/, we'll bring you into the beta as we roll it out to more users, and all will become clear.

        • xMudrii September 22, 2016

          Hello,

          thank you for answer! It explains why there was nothing and why it disappeared.
          I assume it is safe to delete if someone wants to do so.

          In meanwhile, I registred for beta program and hope to be accepted one day so I can help testing new features. ;)

          • eris MOD September 22, 2016

            It's super safe to uninstall, but it's your call. I included some instructions to do so in my answer below.

            I'm actually really surprised people clicked that button without knowing what it does! In this case, it's mostly harmless, but I wouldn't personally install anything on a (non-test) machine without knowing anything about it.

            💙

eris MOD September 22, 2016

Sorry about this. The feature this matches up with is in a closed beta right now. It was publicly viewable for a short time yesterday and this morning, by accident. :(

I'd recommend removing the agent by running one of the following commands depending on which Distro you're running:

sudo apt-get purge do-agent
sudo yum remove do-agent

Reply
  • xMudrii September 22, 2016

    Thanks for clearing it up. :D

    For some odd reason I can't reply to previous comment.

    Well, in my case it was test droplet. My rule no 1 is don't touch non-test server without detailed testing. :D

    I was bit curious and wanted to test something. Anyways I found one bug which I sent to dobeta@ this evening, I hope this address exist, found it reading script ;) :D

donovanh September 22, 2016

I installed it today also, saw a significant rise in CPU usage and data IO also. I don't support you have any idea what this thing is meant to do? I see no option to make use of this data.

Reply
basecamp September 22, 2016

Installed as well, I'm seeing the same thing. Interested to know what exactly it does.

Reply
Have another answer? Share your knowledge.

Related Questions