Question

Managed MongoDB vs self-managed -- will my scripts still crash mongod?

Posted January 11, 2022 157 views
MongoDBDigitalOcean Managed MongoDB Database

Hi there, I’m working on a big data / analytics project, running entirely on a DO droplet (Ubuntu). The DB backend for this project is MongoDB.

So far, I’ve run the project (pre-production) on a MongoDB install on the droplet which I’m maintaining. In the past stages of the project, I’ve had issues where a python script, meant to fill the MongoDB with data, would hit the droplet’s CPU and RAM limits, and end up crashing the mongod process. This would in turn mean that these scripts, all triggered from the crontab wouldn’t run, and data wouldn’t be ingested.

I want a scenario where I needn’t worry about my scripts breaking the MongoDB. In other words, I would like to get a managed DO MongoDB with basic resources, and for me to e.g. get an email if I the CPU/RAM that I’ve provisioned isn’t going to cut it, but for the code still to run and hence fill the database with data. Further, it would be great if this could mean that I could again reduce the resources I have for my droplet.

Thanks in advance for any help with this!

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
Submit an Answer
1 answer

Hi there,

I believe that this would really depend on what resources your scripts require.

I could suggest testing this out for your specific case by spinning up a small MongoDB Managed Cluster and running your scripts to see how they perform. You could shutdown the cluster after the test so that you would only be charged for the couple of hours that you’ve used the cluster for.

Regarding the Droplets, in such a case I would personally add a bit more resources to make sure that there are enough.

Regards,
Bobby

  • Hi Bobby, thanks for the reply. I will try testing with a managed cluster eventually, that makes sense.

    However, in the interim, I’ve doubled the resources of my droplet, continuing with the self-managed version of Mongo. So I now have 4 cores and 8GB Ram.

    I time the scripts in my project that are performing aggregations. Now, this is really weird.
    Completion time (aggregation with $match, $lookup and $project) in old configuration (½ Ram) – 31 secs. In new configuration: 51 secs.
    Monitoring the whole thing on htop, I can see that this never goes beyond 1 of my 4 processor cores, and it also never goes above 3 GB of Ram (for the entire system).

    What’s going on?

    • Hello,

      What I could suggest in this case is to check your server logs for some more information. Feel free to share the logs here too so I could try to advise you further.