Images uploaded using Rails Active Storage to DO Spaces are public, but videos are incorrectly private

Question

I have a Rails 6.1.6.1 app that is using Active Storage to upload images to Digital Ocean Spaces. All images are uploaded with public permissions. This has worked fine for over 2 years. I’ve now been adding video uploads as well, but using the exact same set up, all videos are uploaded as private instead, no matter what I do. Setting the video to public manually through the DO dashboard sets the permissions as expected.

My storage.yml file is set up as follows:

digitalocean:
  service: S3
  access_key_id: <%= ENV["DIGITALOCEAN_SPACES_KEY"] %>
  secret_access_key: <%= ENV["DIGITALOCEAN_SPACES_SECRET"] %>
  region: <%= ENV["DIGITALOCEAN_SPACES_REGION"] %>
  bucket: <%= ENV["DIGITALOCEAN_SPACES_BUCKET"] %>
  endpoint: <%= ENV["DIGITALOCEAN_SPACES_ENDPOINT"] %>
  public: true
  upload:
    cache_control: "public, max-age=31536000"

Where public: true takes care of the permissions correctly for images.

Both images are videos are uploaded using Direct Upload through the same method.

When I upload an image it is send using the following url (with some bits redacted): https://[bucket].fra1.digitaloceanspaces.com/[file-id]?x-amz-acl=public-read&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=[key]%2F20230922%2Ffra1%2Fs3%2Faws4_request&X-Amz-Date=20230922T164911Z&X-Amz-Expires=300&X-Amz-SignedHeaders=content-length%3Bcontent-md5%3Bcontent-type%3Bhost&X-Amz-Signature=[signature]. The important bit (I’m assuming) being x-amz-acl=public-read

When I upload a video the url is similar: https://[bucket].fra1.digitaloceanspaces.com/[file-id]?x-amz-acl=public-read&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=[key]%2F20230922%2Ffra1%2Fs3%2Faws4_request&X-Amz-Date=20230922T165148Z&X-Amz-Expires=300&X-Amz-SignedHeaders=content-length%3Bcontent-md5%3Bcontent-type%3Bhost&X-Amz-Signature=[signature]. The same x-amz-acl=public-read is present, yet it is private.

I’ve tried adding the x-amz-acl header to the accepted headers in the DO Spaces settings. I’ve tried adding public: acl: "public-read" to the storage.yml. I’ve tried manually sending the header with every request. But at this point I feel like I’m fighting against something that should have just worked, in theory.

The images are all png or jpeg, the video is always an mp4. The images are limited to 2MB, but videos can of course be much larger (the smallest I’ve tried is 10MB), could it be related to file size somehow?

Any help would be much appreciated!

Mitcheljager · Accepted Answer

Turns out I was looking for the answer in the wrong place. It wasn’t so much that videos were uploaded as private, all files were. Variants were public, but videos didn’t have variants. I’m still a bit puzzled why this is, since the x-amz-acl=public-read header was included. I am assuming it’s because the variant is processed directly through Rails, where as the original file is uploaded with DirectStorage.

I ended up solving this by monkey patching @rails/activestorage to include the same header in the request. This is the patch (which you can use using the patch-package npm package):

diff --git a/node_modules/@rails/activestorage/app/assets/javascripts/activestorage.js b/node_modules/@rails/activestorage/app/assets/javascripts/activestorage.js
index 6061fb9..63d33cf 100644
--- a/node_modules/@rails/activestorage/app/assets/javascripts/activestorage.js
+++ b/node_modules/@rails/activestorage/app/assets/javascripts/activestorage.js
@@ -560,6 +560,7 @@
       this.xhr.setRequestHeader("Content-Type", "application/json");
       this.xhr.setRequestHeader("Accept", "application/json");
       this.xhr.setRequestHeader("X-Requested-With", "XMLHttpRequest");
+      this.xhr.setRequestHeader("x-amz-acl", "public-read");
       var csrfToken = getMetaValue("csrf-token");
       if (csrfToken != undefined) {
         this.xhr.setRequestHeader("X-CSRF-Token", csrfToken);
@@ -638,6 +639,7 @@
       for (var key in headers) {
         this.xhr.setRequestHeader(key, headers[key]);
       }
+      this.xhr.setRequestHeader("x-amz-acl", "public-read")
       this.xhr.addEventListener("load", function(event) {
         return _this.requestDidLoad(event);
       });
diff --git a/node_modules/@rails/activestorage/src/blob_record.js b/node_modules/@rails/activestorage/src/blob_record.js
index 997c123..f626d5c 100644
--- a/node_modules/@rails/activestorage/src/blob_record.js
+++ b/node_modules/@rails/activestorage/src/blob_record.js
@@ -17,6 +17,7 @@ export class BlobRecord {
     this.xhr.setRequestHeader("Content-Type", "application/json")
     this.xhr.setRequestHeader("Accept", "application/json")
     this.xhr.setRequestHeader("X-Requested-With", "XMLHttpRequest")
+    this.xhr.setRequestHeader("x-amz-acl", "public-read")

     const csrfToken = getMetaValue("csrf-token")
     if (csrfToken != undefined) {
diff --git a/node_modules/@rails/activestorage/src/blob_upload.js b/node_modules/@rails/activestorage/src/blob_upload.js
index 277cc8f..c7c5864 100644
--- a/node_modules/@rails/activestorage/src/blob_upload.js
+++ b/node_modules/@rails/activestorage/src/blob_upload.js
@@ -11,6 +11,7 @@ export class BlobUpload {
     for (const key in headers) {
       this.xhr.setRequestHeader(key, headers[key])
     }
+    this.xhr.setRequestHeader("x-amz-acl", "public-read")
     this.xhr.addEventListener("load", event => this.requestDidLoad(event))
     this.xhr.addEventListener("error", event => this.requestDidError(event))
   }

All in all not the most elegant solution, but after hours of debugging I’m just glad it works.

Question

Images uploaded using Rails Active Storage to DO Spaces are public, but videos are incorrectly private

Try DigitalOcean for free

Popular Topics

Featured on Community

Get our biweekly newsletter

Hollie's Hub for Good

Become a contributor

Featured Tutorials

DigitalOcean Products

Welcome to the developer cloud