How to access all the files stored in a Spaces Folder

In order to be compatible with existing tools, the Spaces API was designed to be inter-operable with the S3 API. With S3, there is no real concept of a folder. When you create a “folder” through the Spaces UI, it is actually just a “key” with a zero sized “object.” Items inside the folder have the folder key as a prefix to their own key.

For example, here is a listing of objects in a Space with a folder named foo containing a file named bar using the aws cli:

$ aws s3api --endpoint-url https://nyc3.digitaloceanspaces.com --profile do list-objects --bucket my-bucket
{
    "Contents": [
        {
            "LastModified": "2017-10-17T17:50:50.840Z", 
            "ETag": "\"d41d8cd98f00b204e9800998ecf8427e\"", 
            "StorageClass": "STANDARD", 
            "Key": "foo/", 
            "Owner": {
                "DisplayName": "681451698", 
                "ID": "681451698"
            }, 
            "Size": 0
        }, 
        {
            "LastModified": "2017-10-17T17:56:08.583Z", 
            "ETag": "\"8cf8463b34caa8ac871a52d5dd7ad1ef\"", 
            "StorageClass": "STANDARD", 
            "Key": "foo/bar", 
            "Owner": {
                "DisplayName": "681451698", 
                "ID": "681451698"
            }, 
            "Size": 2
        }
    ]
}

You can programmatically list the contents of a “folder” by including a prefix query parameter when calling the API to list the contents of a Space. Here’s a quick Python 3 example:

import boto3

session = boto3.session.Session()
client = session.client('s3',
                        region_name='nyc3',
                        endpoint_url='https://nyc3.digitaloceanspaces.com',
                        aws_access_key_id='ACCESSKEY',
                        aws_secret_access_key='SECRETKEY')


resp = client.list_objects(Bucket='my-bucket', Prefix='foo/')
for obj in resp['Contents']:
     print(obj['Key'])