Has anyone used aws-sdk to operate on DO Spaces? How to do it?

Question

I’m trying to wrap my head around this but no luck thus far. The DO Documentation states that the Spaces API is pretty much designed .to be used as S3 in most cases. So i’ve been trying to use aws-sdk for NodeJS, to create buckets in Spaces and send files there.
Here is what I’m trying:

var AWS = require("aws-sdk");
var EP = new AWS.Endpoint("kdsuserdata.nyc3.digitaloceanspaces.com");

fs.readFile(file.path, function (err_file, data) {
          if (err_file) throw err_file; // Something went wrong!
          var s3bucket = new AWS.S3({endpoint: EP, params: {Bucket: result._id}});  //MongoDB User id
          s3bucket.createBucket(function(){
            var params = {Key: file.name, Body: file}; 
            s3bucket.upload(params, function(err_s3, data_s3){
              fs.unlink(file.path, function (err) {
                if (err) {     
                  console.error(err);             
                }              
                console.log('Temp File Delete');
              });              
              if(err_s3) throw err_s3;        

              return res.json({result: data_s3, err: null});
            })                 
          })
       });

With this I’m getting the error below:

/Users/jh/Documents/dash/node_modules/aws-sdk/lib/request.js:31
            throw err;
            ^

Error: Unsupported body payload object
at ManagedUpload.self.fillQueue (/Users/jh/Documents/dash/node_modules/aws-sdk/lib/s3/managed_upload.js:90:21)
    at ManagedUpload.send (/Users/jh/Documents/dash/node_modules/aws-sdk/lib/s3/managed_upload.js:199:33)
    at features.constructor.upload (/Users/jh/Documents/dash/node_modules/aws-sdk/lib/services/s3.js:1067:50)
    at Response.<anonymous> (/Users/jh/Documents/dash/server/api/client/client.controller.js:217:22)
    at Request.<anonymous> (/Users/jh/Documents/dash/node_modules/aws-sdk/lib/request.js:364:18)
    at Request.callListeners (/Users/jh/Documents/dash/node_modules/aws-sdk/lib/sequential_executor.js:105:20)

Any help will be appreciated.

Answer 1

xMudrii September 19, 2017

I don’t know am I late or not, but I played with this and managed to come up with some solution.
Yes, you can use aws-sdk, and it works. Using your code, I ended up with new bucket and with file in it.

I used the following code:

var AWS = require("aws-sdk");
var EP = new AWS.Endpoint("nyc3.digitaloceanspaces.com");

var s3bucket = new AWS.S3({endpoint: EP, params: {Bucket: 'testabc'}}); 
s3bucket.createBucket(function(){
    var params = {Key: 'Test1', Body: 'Hello!'};
    s3bucket.upload(params, function(err_s3, data_s3){
        if(err_s3) throw err_s3;
        console.log(data_s3)
        console.log(err_s3)
    })
})

This created me bucket testabc with file Test1 in it.
As you can see from your code, you made mistake in EP variable.
The endpoint format for bucket is {BUCKET}.{REGION}.digitaloceanspaces.com. As you are creating new bucket, you should drop it from the endpoint, so you will have {REGION}.digitaloceanspaces.com.

Reply Report

mtndesign September 21, 2017

Thank you @xMudrii.

I wanted to add a folder to my existing bucket ‘kdsuserdata’ for each user, and add their datas (photos mostly) to that folder. Now I understood what I’m doing wrong. Thank you for pointing me to the right direction. I’m going to post new version of my code, which’ll create new folder in an existing bucket and add files to it, using aws-sdk.

Have a nice day!
Reply Report

Answer 2

mtndesign September 21, 2017

Thank you @xMudrii.

I wanted to add a folder to my existing bucket ‘kdsuserdata’ for each user, and add their datas (photos mostly) to that folder. Now I understood what I’m doing wrong. Thank you for pointing me to the right direction. I’m going to post new version of my code, which’ll create new folder in an existing bucket and add files to it, using aws-sdk.

Have a nice day!
Reply Report

Answer 3

huucuong1612 September 21, 2017

bucket.nyc3.digitaloceanspaces.com/437185df-27b3-4c4b-b56d-2b36e255a9f6jpg_1505980933677.jpg. Response for preflight has invalid HTTP status code 403

I Got the problem 403, someone can help

Reply Report

mtndesign September 21, 2017

I’d check the ACL (Permissions) for that resource. Not the bucket, but the file in it. HTTP403 is “Access Forbidden”. Preflight is HTTP OPTIONS, or sometimes HEAD request to check for CORS. If your resource (the file in the bucket) has private access permission, you cannot access it publicly, you need to be authenticated, and authorized.

P.S: I couldn’t test this URL right now, it returns HTTP 404.
Reply Report
- huucuong1612 September 21, 2017
  
  change bucket to actiontwo, actiontwo.nyc3.digitaloceanspaces.com/437185df-27b3-4c4b-b56d-2b36e255a9f6jpg_1505980933677.jpg. Response for preflight has invalid HTTP status code 403
  
  I add the secret key and access key, but it response the 403 code, it cause by CORS, could you help me how to config that ,
  When i use s3 AWS, i know where to config but digitalocean i don’t know where
  
  Reply Report
- huucuong1612 September 21, 2017
  
  the actiontwo.nyc3.digitaloceanspaces.com/437185df-27b3-4c4b-b56d-2b36e255a9f6jpg_1505980933677.jpg is the key i prepare for upload but it got 403 when i try to upload, i used aws-sdk
  
  my config
  
  config ={ endpoint : 'nyc3.digitaloceanspaces.com' accessKeyId : creds.access_key secretAccessKey : creds.secret_key } AWS.config.update config AWS.config.region = 'NYC3' bucket = new (AWS.S3)({ params: Bucket: creds.bucket } )
  
  edited by asb
  
  Reply Report

Answer 4

mtndesign September 21, 2017

I’d check the ACL (Permissions) for that resource. Not the bucket, but the file in it. HTTP403 is “Access Forbidden”. Preflight is HTTP OPTIONS, or sometimes HEAD request to check for CORS. If your resource (the file in the bucket) has private access permission, you cannot access it publicly, you need to be authenticated, and authorized.

P.S: I couldn’t test this URL right now, it returns HTTP 404.
Reply Report
- huucuong1612 September 21, 2017
  
  change bucket to actiontwo, actiontwo.nyc3.digitaloceanspaces.com/437185df-27b3-4c4b-b56d-2b36e255a9f6jpg_1505980933677.jpg. Response for preflight has invalid HTTP status code 403
  
  I add the secret key and access key, but it response the 403 code, it cause by CORS, could you help me how to config that ,
  When i use s3 AWS, i know where to config but digitalocean i don’t know where
  
  Reply Report
- huucuong1612 September 21, 2017
  
  the actiontwo.nyc3.digitaloceanspaces.com/437185df-27b3-4c4b-b56d-2b36e255a9f6jpg_1505980933677.jpg is the key i prepare for upload but it got 403 when i try to upload, i used aws-sdk
  
  my config
  
  config ={ endpoint : 'nyc3.digitaloceanspaces.com' accessKeyId : creds.access_key secretAccessKey : creds.secret_key } AWS.config.update config AWS.config.region = 'NYC3' bucket = new (AWS.S3)({ params: Bucket: creds.bucket } )
  
  edited by asb
  
  Reply Report

Answer 5

dusblinov September 22, 2017

Hello! I also try to configure the space instead of AWS, as far as I can see that authorization does not work. I get the error 403. How to sign the header in this case?
Do I understand correctly that I need to rewrite everything to aws4 authorization?

AWS.config.accessKeyId = settings.aws.AWS_ACCESS_KEY_ID;
AWS.config.secretAccessKey = settings.aws.AWS_SECRET_ACCESS_KEY;

const bucketName = settings.aws.S3_BUCKET;
const EP = new AWS.Endpoint("nyc3.digitaloceanspaces.com");
const s3bucket = new AWS.S3({ endpoint: EP });

const s3Params = {
    Bucket: bucketName,
    Key: fileName,
    Expires: 60,
    ContentType: file.type,
    ACL: 'public-read',
};

s3bucket.getSignedUrl('putObject', s3Params, (err, data) => {
    console.log(err, data)
    if (err) {
        debug('image loading error', err);
    }   else {
        const returnData = {
            requestUrl: data,
            imageUrl: `${settings.cdn.full}${fileName}`,
        };
        res.json(ImageResponse(returnData));
    }
});

Reply Report

Answer 6

Caashiere September 30, 2017

[deleted]

Reply Report

Related

Question

Has anyone used aws-sdk to operate on DO Spaces? How to do it?

Related

Leave a comment

Looking for something else?

Sign up for our newsletter

Related

Question

Has anyone used aws-sdk to operate on DO Spaces? How to do it?

Related

Leave a comment

Related

question

This site can’t be reached Nodejs backend server

question

How to access my API nodejs

question

cpu from 13% to 100% in one minute

question

Problem installing wordpress on subdomain (nginx / ubuntu)

Looking for something else?