Related

Product DigitalOcean Spaces: Simple Object Storage Product
Droplet resize not working Question Question
Failed to create new system journal: No space left on device Question Question

Question

Which DO product should we use for storing millions of image files?

Posted March 9, 2020 911 views
DigitalOcean SpacesDigitalOcean Volumes

Hi there,

Which solution is a better fit for an App that overtime will reach millions of image files? Already migrating to Spaces, but does Spaces support/handle 10’s of millions of objects ? Maybe even distributed across many spaces/buckets ? I’m already at 2.6m objects for my 150k “entities”. My total entities are 4m. Should I continue the migration or seek a different solution?

Add a comment
petsoukos
By:
petsoukos

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
1 answer
alexdo March 10, 2020

Hello, @petsoukos

The DigitalOcean’s Spaces should be suitable with your needs. However please consider the fact that one space object has 250GB storage and include 1,024 GiB of outbound data transfer (from Spaces to the internet), which is shared between all Spaces, so it is important to keep in mind that there are rate limits.

You can also contact our support team about your inquiry and you can further discuss your plan and needs.

I will suggest you to check our official documentation about the product in order to make sure that this is what you need.

If you’re interested about the Limits of the Spaces you can check:

https://www.digitalocean.com/docs/spaces/#limits

Here is a link to the official Docs

Hope this helps!

Regards,
Alex

Reply Report
  • petsoukos March 10, 2020

    I read somewhere that there is an upper limit in spaces capacity per region.

    Reply Report
    • alexdo March 11, 2020

      Hello, @petsoukos

      All the official rate limits are mentioned on the docs page I’ve sent earlier. If you have any further questions or concerns you can always contact our amazing support team and they will assist you.

      Regards,
      Alex

      Reply Report
      • petsoukos March 11, 2020

        Hey, @alexdo

        I’ve already read through that, thanks.

        Those limits are request limits. Put, Delete, etc.

        I read also somewhere (in guides/tutorials of DO) that there is also a limit on the number of spaces per region and the max capacity of a space (unless the max capacity was mentioned as a limit for block storage).

        I also can’t find some details about the number of objects that can be stored per Space/Region. My app will require a large number of objects. Already in the millions, will reach soon 10’s of millions.

        Spaces GB cost and bandwidth/transfer cost is not an issue for the app, as long as Spaces can support this huge number of objects. I just don’t want to finish this migration only to find out that I hit some max objects limit (as I did when I reached inodes limit).

        I could also spread to different EU Spaces region, but so far only FRA1 is available and I have made 3 spaces/buckets there on FRA1, and spread my objects (image files) evenly across those spaces.

        Reply Report
Submit an Answer

Related

Looking for something else?

Ask a new question Search for more help