talgat7
By:
talgat7

Out of memory kill process

October 26, 2017 145 views
DigitalOcean

I am on 8GB RAM plan, yet facing difficulties in reading 4GB file into the python pandas dataframe. Getting out of memory error. Already allocated 4GB swap file, yet issue is not solved.

Could you pls. support?

2 Answers

What is the error? What does free -m show?

  • [64152.869711] Out of memory: Kill process 7499 (python3) score 972 or sacrifice child

    [64152.874659] Killed process 7499 (python3) total -vm: 12671396kB, anon-rss:79017

    free - m

    Mem:
    total: 7.8G
    used: 4.3M
    free: 3.5G
    shared: 2.7M
    buffers: 8.1M
    cached: 38M

    Swap:
    total: 4.0G
    used: 2.9G
    free: 1.1G
    shared: -
    buffers:-
    cached: -

Wow, 4GB single file reading directly in python on 8GB system? I would extend the swap space to at least 6GB (mkswap I mean) then try to ask kernel to use swap efficiently by modifying swappiness:
Read it here: https://askubuntu.com/questions/103915/how-do-i-configure-swappiness

Have another answer? Share your knowledge.