Receiving 403 from Spaces when accessing private file containing whitespace using AWS SDK

August 1, 2019 229 views
CDN DigitalOcean

I am experiences problems accessing files on DigitalOcean Spaces in situations where the target file contains whitespace in its name. The below function works perfectly for private files without whitespace in the filename.

I am using .Net Core 2.1 and the AWSSDK.S3 package for access through the S3 API.

My code for retrieval (which works for all files containing no whitespace):

public async Task<Stream> GetReadStream()
{
    // Placed here for completeness
    this.client = new AmazonS3Client(options.Key, options.Secret, new AmazonS3Config { ServiceURL = options.Url });

    MemoryStream result = new MemoryStream();
    try
    {
        GetObjectRequest request = new GetObjectRequest
        {
            BucketName = options.Bucket,
            Key = "folder/file with space.zip"
        };
        using (GetObjectResponse response = await client.GetObjectAsync(request))
        using (Stream responseStream = response.ResponseStream)
        {
            responseStream.CopyTo(result);
        }
    }
    catch (AmazonS3Exception e)
    {
        Console.WriteLine("Error encountered ***. Message:'{0}' when reading an object", e.Message);
    }
    catch (Exception e)
    {
        Console.WriteLine("Unknown encountered on server. Message:'{0}' when reading an object", e.Message);
    }
    result.Seek(0, SeekOrigin.Begin);
    return result;
}

From the log I can see that the AWS SDK URL encodes the filename to: “file%20with%20space.zip” and the resulting URL is identical to that found by browsing the file through the DO Dashboard and copying an URL from the file directly.

Reading this post I suspect the problem happens to any private file containing any character needing URL encoding.

Have I done something wrong or is there any solution to this problem apart from avoiding filenames requiring URL encoding?

Thank you in advance for your time.

1 Answer
efox MOD August 1, 2019
Accepted Answer

Hey there,

One thing I recommend trying: explicitly use v2 signatures, as the encoding rules differ from v4 signatures.

Should you continue seeing issues, we’ll need to dig in further as there is a lot of variation from SDK to SDK. Could you capture the full request + response headers that are being sent to the Spaces API by your script so we can evaluate them? With the headers, we’ll be able to tell if this is an issue with encoding on the client SDK end or on our end with Spaces resolving the request.

If you’d prefer, you can write these into a ticket: https://do.co/new-ticket where our Developer Support team can assist (just reference this community post, if you’d like). Or you can include the outputs here, just include them with redacted secrets, etc.

Regards,
Ethan Fox
Storage Developer Support Engineer II - DigitalOcean

  • Thank you Ethan, your assumption was spot on. The SDK became compatible when modifying the initialization code to this:

    var config = new AmazonS3Config { ServiceURL = options.Url, SignatureVersion = "v2" };
    this.client = new AmazonS3Client(options.Key, options.Secret, config);
    AWSConfigsS3.UseSignatureVersion4 = false;
    

    Defining the SignatureVersion AND changing the static flag UseSignatureVersion4 to false has allowed me to get the objects I experienced problems with.

Have another answer? Share your knowledge.