Question

s3cmd does not sync down as expected with Digital Ocean Spaces

Posted on August 17, 2024
Storage DigitalOcean Spaces Object Storage Space Object Storage
Asked by Richard Leishman

I have an issue whereby a few features of s3cmd do not work with DO Spaces, but work as expected with Amazon S3.

When I run the following command it downloads the entirety of the bucket as expected:

s3cmd sync -v --delete-removed "s3://bucket" "bucket"

When I run it a second time:

It downloads the entirety of the bucket again, where it should only download changed files.
It does not delete files in the destination that no longer exist on the source (–delete-removed).

I have ran the command with --debug and the only thing I can see is that DO Spaces always comes back with a file modified date/time of the current date/time.

I assume that s3cmd is using the md5 hash and the modified date/time to figure out if a file has changed since it’s last sync.

Any help would be appreciated.

Submit an answer

This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

Richard Leishman • August 17, 2024

Accepted Answer

After further work to diagnose the issue, I have found a resolution to both issues.

I discovered that DO and AWS act differently with trailing slashes.

Adding trailing slashes fixed this issue on DO Spaces.