Say I have a DO Spaces bucket called my-test-bucket.
When I upload to Spaces using the Python Boto3 library’s upload_fileobj method where Bucket=“my-test-bucket” and Key=“projects/1/data/1/testfile.txt” then the resulting full file path that actually gets created/shown in the Spaces dashboard is /my-test-bucket/my-test-bucket/projects/1/data/1/testfile.txt
Here is the URL that would be shown when I find the item from the DO Spaces dashboard: https://my-test-bucket.sfo3.digitaloceanspaces.com/my-test-bucket/projects/1/datasets/1/testfile.txt
This is breaking my subsequent list_files_v2 calls later on that are passing Bucket=‘my-test-project’ and Prefix=‘projects/1/data/1/’ as arguments.
For additional context, I have double-checked my bucket name and file key arguments to ensure I’m not accidentally putting the bucket name into the file key. I also do not have any periods in my bucket name.
Would appreciate any assistance here as I though I was going crazy until I saw someone having a similar issue here.
I have tested my application using the exact same Boto3 commands except running against MinIO (self-hosted S3 compatible storage server) instead of DO Spaces, and I am getting the desired behavior without the prepended bucket name, which leads me to believe that the issue is with DO Spaces rather than the commands my app is issuing or the Boto3 library.
I’m not sure if the issue is with the uploads, or with the file listing. It’s almost as if the list_files_v2 is expecting the bucket_name in both the bucket argument as well as part of the prefix to the file.
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.
Hey!
I’ve just tested this with the following simple script and it seems to have worked as expected:
The file was uploaded to the destination directory without having the bucket name prepended two times:
Would you mind sharing an example code snippet that you are using so I could further look into this?
- Bobby