I am on 8GB RAM plan, yet facing difficulties in reading 4GB file into the python pandas dataframe. Getting out of memory error. Already allocated 4GB swap file, yet issue is not solved.
Could you pls. support?
[64152.869711] Out of memory: Kill process 7499 (python3) score 972 or sacrifice child
[64152.874659] Killed process 7499 (python3) total -vm: 12671396kB, anon-rss:79017
free - m
Mem:
total: 7.8G
used: 4.3M
free: 3.5G
shared: 2.7M
buffers: 8.1M
cached: 38M
Swap:
total: 4.0G
used: 2.9G
free: 1.1G
shared: -
buffers:-
cached: -
Wow, 4GB single file reading directly in python on 8GB system? I would extend the swap space to at least 6GB (mkswap I mean) then try to ask kernel to use swap efficiently by modifying swappiness:
Read it here: https://askubuntu.com/questions/103915/how-do-i-configure-swappiness