I have an issue whereby a few features of s3cmd do not work with DO Spaces, but work as expected with Amazon S3.
When I run the following command it downloads the entirety of the bucket as expected:
s3cmd sync -v --delete-removed "s3://bucket" "bucket"
When I run it a second time:
It downloads the entirety of the bucket again, where it should only download changed files.
It does not delete files in the destination that no longer exist on the source (–delete-removed).
I have ran the command with --debug and the only thing I can see is that DO Spaces always comes back with a file modified date/time of the current date/time.
I assume that s3cmd is using the md5 hash and the modified date/time to figure out if a file has changed since it’s last sync.
Any help would be appreciated.
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
Accepted Answer
After further work to diagnose the issue, I have found a resolution to both issues.
I discovered that DO and AWS act differently with trailing slashes.
Adding trailing slashes fixed this issue on DO Spaces.
"s3://bucket" "bucket"
to:
"s3://bucket/" "bucket/"
Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.
Full documentation for every DigitalOcean product.
The Wave has everything you need to know about building a business, from raising funding to marketing your product.
Stay up to date by signing up for DigitalOcean’s Infrastructure as a Newsletter.
New accounts only. By submitting your email you agree to our Privacy Policy
Scale up as you grow — whether you're running one virtual machine or ten thousand.
Sign up and get $200 in credit for your first 60 days with DigitalOcean.*
*This promotional offer applies to new accounts only.