Question

Inode limit, how to increase inode limit on block storage volume?

Posted December 23, 2018 7.6k views
DigitalOceanLinux CommandsBlock StorageStorageUbuntu 18.04

I’ve successfully added a 100gb block storage volume to my droplet, which was originally added to store a few million .json files and a few million image files. The issue I am noticing at the moment is that after the first 3 million .json files the inode used % is already at 85%, even though the storage usage is only at 12%.

seen here: https://imgur.com/a/NEM0e3i

What I am looking to do is reformat the volume and hopefully adjust the inode table to better fit my 100gb disk space and the files I am looking to store. In total I am looking to store around 8-10 million small .json files and around 4-5 million image files.

If possible I am looking for some guidance/how to on how to adjust the inode table to allow storage of these millions of files, ideally from the digitalocean console command line.

Currently I am trying to avoid my default solution of just increasing the volume size to something like 500gb just to increase the inode limit and end up in total using 20% of the 500gb storage and paying 5x the cost.

I am not a server admin in general so most of this is learn as you go/trial and error,

Any help or insight would be appreciated,

Thanks

-Matt

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
1 answer

How small/large are these files? You can set the number of inodes at format-time of the block storage. It can be as high as 4 billion inodes. So partition the new the block storage as ext4 with parted then format it with this additional parameter:

mkfs.ext4 -N 4000000000 /dev/path/to/volume

Then mount it as you mount any volume and rsync the existing json files. I wouldn’t create it with a capacity of 4 billions inodes as that’s a bit extreme. It also depends on the average size of the json files. Maybe set it to 1 or 2 billion max.

Cheers

Submit an Answer