manossef
By:
manossef

Valgrind can't run: out of memory?

January 16, 2015 3.9k views

I try to run a simple test with valgrind but it seems that valgrind can't even start because of some memory limitation issue.

Command:

valgrind --leak-check=full --track-origins=yes echo 'test'

Output:

==26260==     Valgrind's memory management: out of memory:
==26260==        initialiseSector(TC)'s request for 27597024 bytes failed.
==26260==        143327232 bytes have already been allocated.
==26260==     Valgrind cannot continue.  Sorry.

Only 355/490MB of my total RAM is consumed so I would guess that is not the reason behind this since valgrind is such a small tool. Also I run the test inside a docker container if that matters.

Any idea on how to solve?

6 comments
  • To eliminate some kind of system limit as the culprit, can you look at the output to the following from a docker container:

    ulimit -a | grep memory
    
  • I looked at that actually but my virtual memory looks fine. Not sure if there is some other kind of memory to look at but here it is:

    root@box:~# ulimit -a | grep memory
    max locked memory       (kbytes, -l) 64
    max memory size         (kbytes, -m) unlimited
    virtual memory          (kbytes, -v) unlimited
    
  • @manossef - What was the '-m' parameter you gave your docker run command?

  • Hi, I am not sure what you mean. Btw the command you gave gives me the same output both in the terminal of the server (which is a docker image) and an own Docker image. I noticed that on my personal computer this issue is not present, it's only on the server at Digital Ocean.

    On a side note I was getting the error message mentioned when trying to run valgrind in my already image. I tried to run valgrind in a totally new docker image and I still have problems but this time instead I get

    ==231== Memcheck, a memory error detector
    ==231== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
    ==231== Using Valgrind-3.10.0.SVN and LibVEX; rerun with -h for copyright info
    ==231== Command: echo test
    ==231==
    Killed

    So while before it gave an error, now it doesn't run at all (in a different image).

  • Here is a way to reproduce the problem on a server:

    root@box:~# docker run -it ubuntu bash
    root@09c035bf2786:/# apt-get update -qq && apt-get install -qqy valgrind
    root@09c035bf2786:/# valgrind --leak-check=full --track-origins=yes echo 'test'
    ==228== Memcheck, a memory error detector
    ==228== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
    ==228== Using Valgrind-3.10.0.SVN and LibVEX; rerun with -h for copyright info
    ==228== Command: echo test
    ==228==
    Killed
    

    So it doesn't even run..

  • @manossef - When you run docker, you can give it a --memory or -m parameter for the maximum memory to use. For example, to give a container a maximum of 375Mb memory

    docker run --memory=375m  -it ubuntu bash
    
1 Answer

One thing that it important to note here is that running a program with valgrind will require about twice as much memory as normal usage. This is because it keeps around a shadow copy of allocated memory.

The section on valgrind in this blog post about debugging tools was really helpful for me in understanding what it is up to underneath the hood:

To do this, the first thing memcheck does is to maintain a registry of all allocated memory. Every time a new chunk memory is allocated memcheck keeps track of it by remembering the returned pointer, the amount of memory allocated as well as the backtrace from which it has been allocated. Additionally, it adds some redzones around the allocated memory that cannot be allocated in order to easily detect out-of-buffer accesses.

Needless to say it will also catch every single deallocation in order to keep its registry up to date. The deallocation does not immediately remove the entry from the registry, it marks it as deallocated and remembers the deallocation backtrace. By putting deallocated memory in quarantine, it ensures that use-after-free accesses can be caught as such since that memory cannot be reused for other purpose too rapidly.

Have another answer? Share your knowledge.