I’ve successfully added a 100gb block storage volume to my droplet, which was originally added to store a few million .json files and a few million image files. The issue I am noticing at the moment is that after the first 3 million .json files the inode used % is already at 85%, even though the storage usage is only at 12%.
seen here: https://imgur.com/a/NEM0e3i
What I am looking to do is reformat the volume and hopefully adjust the inode table to better fit my 100gb disk space and the files I am looking to store. In total I am looking to store around 8-10 million small .json files and around 4-5 million image files.
If possible I am looking for some guidance/how to on how to adjust the inode table to allow storage of these millions of files, ideally from the digitalocean console command line.
Currently I am trying to avoid my default solution of just increasing the volume size to something like 500gb just to increase the inode limit and end up in total using 20% of the 500gb storage and paying 5x the cost.
I am not a server admin in general so most of this is learn as you go/trial and error,
Any help or insight would be appreciated,
Thanks
-Matt
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
Accepted Answer
How small/large are these files? You can set the number of inodes at format-time of the block storage. It can be as high as 4 billion inodes. So partition the new the block storage as ext4 with parted then format it with this additional parameter:
mkfs.ext4 -N 4000000000 /dev/path/to/volume
Then mount it as you mount any volume and rsync the existing json files. I wouldn’t create it with a capacity of 4 billions inodes as that’s a bit extreme. It also depends on the average size of the json files. Maybe set it to 1 or 2 billion max.
Cheers
Become a contributor for community
Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.
DigitalOcean Documentation
Full documentation for every DigitalOcean product.
Resources for startups and SMBs
The Wave has everything you need to know about building a business, from raising funding to marketing your product.
The developer cloud
Scale up as you grow — whether you're running one virtual machine or ten thousand.
Get started for free
Sign up and get $200 in credit for your first 60 days with DigitalOcean.*
*This promotional offer applies to new accounts only.