I am on 8GB RAM plan, yet facing difficulties in reading 4GB file into the python pandas dataframe. Getting out of memory error. Already allocated 4GB swap file, yet issue is not solved.
Could you pls. support?
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
Wow, 4GB single file reading directly in python on 8GB system? I would extend the swap space to at least 6GB (mkswap I mean) then try to ask kernel to use swap efficiently by modifying swappiness: Read it here: https://askubuntu.com/questions/103915/how-do-i-configure-swappiness
Hello,
I’ll recommend to examine the logs if the droplet us running out of memory. Usually there will be references in /var/log/messages indicating that a process was killed - oom or out of memory
What I would usually do if I’m getting that error on a Droplet with 1GB of RAM, is to add swap space, that way you would have some extra buffer in case that your RAM is fully exhausted. This step usually benefits a lot a smaller droplet.
You can follow the steps on how to do that here:
https://www.digitalocean.com/community/tutorials/how-to-add-swap-space-on-ubuntu-22-04
Regards
Become a contributor for community
Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.
DigitalOcean Documentation
Full documentation for every DigitalOcean product.
Resources for startups and SMBs
The Wave has everything you need to know about building a business, from raising funding to marketing your product.
The developer cloud
Scale up as you grow — whether you're running one virtual machine or ten thousand.
Get started for free
Sign up and get $200 in credit for your first 60 days with DigitalOcean.*
*This promotional offer applies to new accounts only.