Cannot find what is filling up disk space /dev/vda1 is 100% full

August 14, 2014 14.3k views

Here is what I have:

df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda1        40G   38G     0 100% /

However, I cannot find what is using the space up with du -sh *

Any ideas how to find what is using up all that space?

  • I'm having the same issue. Did you find a resolution?

  • Same issue here!

  • I am also seeing this

    du -csh /
    1.6G    total


    Filesystem      Size  Used Avail Use% Mounted on
    /dev/vda1        30G   28G     0 100% /
  • After sending a support ticket about this issue, they point me to a direction of solving it myself.

    For me there's actually a log file that stacked up until the disk is full.
    Here's how I solve it:
    I go to root dir: cd /
    list dirs and files: ls
    And do du -h [dir] for each of them
    When you see something suspicious you go inside that dir, or see from the du result.
    For me it's a du -h home
    When I trace, it points me to my laravel app storage/logs.

    On a sidenote, on my issue here is caused by job queue php artisan. And on those job queue there's an error and it keeps logging and repeating, the php process also hogging the cpu. This solved me two issues altogether.

    There you go. I hope it helps.

  • @josualeonard
    Hello, I ve sent a request as well to them, and similar to fix it your self with those links .....

    I don't think I have as much knowledge as you do, can you explain to me in a bit more detail how to solve this ?

    Problem is the same :

    Filesystem Size Used Avail Use% Mounted on
    /dev/vda1 30G 28G 0 100% /

    I am not quite sure what you mean by cd / ect.. ect... It was all working fine until yesterday.
    Thanks for your help !!

8 Answers

Another option that might help you find out what is using up space is ncdu Install it with:

sudo apt-get install ncdu

If you still can't find anything, try running:

lsof / | grep deleted

Sometimes if a file is deleted while it is opened by a process, it will not actually free up the disk space until the process is ended. This command will show if there are any files in that state.

Thank you for responding.

ncdu shows only the disk usage under /home, apparent size 922.3MiB

lsof / | grep deleted shows 8 files of about 4K each.

df -h is still showing

Filesystem Size Used Avail Use% Mounted on
/dev/vda1 40G 38G 0 100% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
udev 991M 8.0K 991M 1% /dev
tmpfs 201M 380K 200M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 1001M 0 1001M 0% /run/shm
none 100M 0 100M 0% /run/user

Something is still in /dev/vda1 for 38G that i cannot find

I think you should run ncdu against root directory

sudo ncdu /

Googled and found this question which had the same problem.

@TonyTsang That command helped me find what it was

Apparently it was a hidden log folder set by pm2.

Used ncdu / to check what was using the pace, apparently like 15Gb of mail logs.

I deleted them to clear space, and a day later I got 700Mb log of mail. Considering I'm only hosting 2 wordpress blogs, seems like I got some spam issue going on :((

I'll open another thread for this, because I have no idea how to solve it.

  • Do you have any cron jobs set up? If so, you can disable email notifications by running crontab -e and:

    • adding the following line at the top of the file to disable email notifications for all cronjobs:
    • or, adding >/dev/null 2>&1 to each cronjob like this, which will redirect all output to /dev/null:
    5 10 * * * /path/to/my/script >/dev/null 2>&1
    • There are no cron jobs set up. I checked the mail queue and there were like 4200 spam messages trying to be sent.

      I managed to find all the infected files in the wordpress installation, delete them, clear the mail queue, restart the mail server and it seems to be working fine. Some days later, the queue is still empty, and the server usage looks normal. I guess that was an out to date wordpress plugin :S


outside of other answers here what I did was:

navigate through subfolders with ncdu and finding large folders. Among them one can delete:

https://bbs.archlinux.org/viewtopic.php?id=158510 // /var/logs/journal
http://askubuntu.com/questions/100004/how-can-i-free-space-from-a-massive-39-5gb-var-log-folder // other already archived (GZ-ipped) files;

and compress .git folders of projects by shrinking them with this command: git repack -a -d --depth=250 --window=250

Hi @BreizhoO
I thought it wasn't related to my code, I thought it might be a server problem that create new partition. Apparently I'm wrong, that /dv/vda1 / is the main partition, only this time filled with some huge files, the log files.

First, stop any queue or running cron job as it might be the cause your log files is full.
I solve this by removing some log files that keeps building up when there's an error, and that error was caused by a failed job queue which is keep repeating (keep trying to run the job) and store an error log line everytime it runs. I wasn't aware until my disk was full and got a notification from newrelic.

You can follow my instructions there. Imagine you're going into a room and check the content of each room. :) Basically I'm just trying to find which dir is it. That's why I'm starting from the very top.
Go to root dir by cd /
List dirs and files: ls so you can go to each one of it.
Example results:


And do du -h [dir] for each of them:

du -h /usr
du -h /var
du -h /home <-- suspicious, contains couple of GBs of size

From each one of it see which one is suspicious (with largest dir size), that's why I said mine was on /home
Then I'm doing cd /home, do the same from here: ls, do du -h /home/vagrant
Then I'm doing cd /home/vagrant, do the same from here: ls, do du -h /home/vagrant/mylaravelproject
Then I'm doing cd /home/vagrant/mylaravelproject, do the same from here: ls, do du -h /home/vagrant/mylaravelproject/storage
Then I'm doing cd /home/vagrant/mylaravelproject/storage, do the same from here: ls, do du -h /home/vagrant/mylaravelproject/storage/logs
Until you finally sure that directory was the culprit.
For me it was on /home/vagrant/mylaravelproject/storage/logs
On that /logs dir there was a huge 8 to 15gb of log files X_x
I remove it.. and voila!

Does it makes sense? Some people here might have better way of finding it though.

Have another answer? Share your knowledge.