Question
Slow upload files to DO space bucket
- Posted on August 25, 2023
- DigitalOcean Spaces
Asked by whyyouslap
Hello, I am converting pdfs to jpegs using cloudconvert then uploading these jpegs to a DO space. My server, DO space and db are all in the same zone - NYC3. My server is a dedicated w/ 16GB of RAM and 300GB SSD. CPU load and memory use is low.
Cloudcovert conversion takes a few seconds but uploading ten or fifteen 250kb images to my DO space takes over 3 minutes. My upload speed is 8MB/s.
I am using the python statement s3.upload_fileobj(response.raw, ‘space’, filename) if that helps
Why is uploading to DO space so slow?
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.
Featured on Community
Get our biweekly newsletter
Sign up for Infrastructure as a Newsletter.
Hollie's Hub for Good
Working on improving health and education, reducing inequality, and spurring economic growth? We'd like to help.
Become a contributor
Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.
Hello @whyyouslap
It’s unusual for your upload times to be this slow, especially considering that your server, database, and DO Space are in the same data center (NYC3). Given your specifications and conditions, it indeed appears that you should be experiencing faster upload speeds.
You can use tools like
mtrorpingto measure network latency to your Space.You could also try using a different S3 SDK client to see if performance improves or at least if the upload speed changes so you could have another referencer point.
Hope that this helps!