On a review on running a redis docker container which resulted in this situation, it appears that UFW does not interoperate with Docker well on the Docker on Ubuntu image built from the upstream packages for UFW and Docker.

Docker inserts the rules for forwarding exposed ports in the DOCKER chain, which is included in the FORWARD chain before any UFW rules are processed, so containers that have exposed ports that rely on firewall isolation are not in fact isolated.

The upstream package that Docker is installed from includes a service file that does not pull in options from the /etc/default/docker file, so making changes such as –iptables=false there have no effect. You can mask or update the service unit file to include this option and restart docker to avoid having Docker populate iptables, and manage exposing the ports, though this may have other implications with respect to the iptables rules Docker sets for inter-container communication.

Reviewing the docker logs for the container, it shows that remote connections are made to the redis instance, and without securing through other means ( e.g. requiring authentication ) the connections eventually load a module and configure redis to sync as a slave which is involved in the mechanism for getting the unwanted program from executing on the redis instance.

The id of the process running within the container overlaps with the id of the do-agent user on the host, so from the host, it looks like the process is running as the do-agent user, but in fact is running within the container which isn’t aware of the do-agent user but shares the same id.

We’re looking into this to see what options we have to better assist in surfacing the importance of using secured deployments in containers as well as exploring options to address the UFW/Docker situation. For the moment, I would recommend always using authentication based configurations, and if you rely on the firewall isolation, you will want to modify the Docker service unit to disable modifying iptables, and manage iptables manually for the containers you deploy.

I’m also getting 100% CPU utilization via a do-agent owned task /tmp/kdevtmpfsi. I am using the DO Docker image which is Ubuntu 18.04. Is this related to the use of the metrics agent?

Hi @padsundar,

Have you by any chance installed docker and used it with some images? I’ve seen such behavior on other servers, not explicitly DigitalOcean droplets with the same behavior?

Regards,
KDSys

Yes. I think digital should stop metrics collection service..

I have the same problem with this process /tmp/kdevtmpfsi, it likes come from nowhere… I installed some wordpress theme (also build from docker images) and this mining process comes up. I can’t figure out where is it come from, anyone knows ?

Update:
I found out it was in the volume of redis, alpine

Not sure if metrics collection agents are using docker internally.. Also not sure how it got in first place since iptables were there for last 4 yrs. But yes you are right. The malware clears the iptables and puts an entry in crontab..