How to upload to S3 after getting the signed Url from the backend? #327

harishankards · 2018-05-13T13:16:26Z

Hi,
In my project, I'm using nodejs for backend API and vue for the frontend. I'm using this vue-dropzone in my project.

So, when a file is successfully uploaded into the frontend, I'm taking the filename and type to the backend to create a signedUrl to upload the files in AWS S3. Please refer to the code below.

dropzoneOptions: {
          url: 'http://localhost:3000/attachments', //I'm sending the req to my backend to get the signedUrl
          thumbnailWidth: 150,
          maxFilesize: 0.5,
          headers: { 'Authorization': 'Bearer ' + this.token },
          addRemoveLinks: true,
          dictDefaultMessage: "<i class='fa fa-cloud-upload'></i>UPLOAD ME"
        }

and the success function

vsuccess (file, response) {
        console.log('it is success', file, response) // this response has the signedUrl from my backend       
      }

Now, that I have the signedUrl here, how do I send the file to the S3?

In the example provided by you here, the signedURL seemed to be put in a hardcoded way.
But, I'm doing in a dynamic way. So, how do I achieve that?

Thanks in advance.

vrajroham · 2018-05-14T07:04:07Z

Hi @harishankards ,

The signedURL seemed to be put in a hardcoded way

The signed URL you found documentation page is sample response and that too is generated dynamically using this php script.

I'm doing in a dynamic way. So, how do I achieve that?

As per my understating, You are getting signed url from server endpoint http://localhost:3000/attachments and you want to upload your file to AWS S3 using that pre-signed url. right?

You need to add additional prop awss3 to dropzone as below,

<vue-dropzone 
      :awss3="awss3"
      v-on:vdropzone-s3-upload-error="s3UploadError"
      v-on:vdropzone-s3-upload-success="s3UploadSuccess">
</vue-dropzone>

Add awss3 prop with dropzoneOptions as below,

dropzoneOptions: {
      url: 'http://localhost:3000/file-upload', // File upload endpoint, not signed url endpoint
      thumbnailWidth: 150,
      maxFilesize: 0.5,
      headers: { 'Authorization': 'Bearer ' + this.token },
      addRemoveLinks: true,
      dictDefaultMessage: "<i class='fa fa-cloud-upload'></i>UPLOAD ME"
},
awss3: {
      signingURL: 'http://localhost:3000/attachments', //Where you will get signed url
      headers: {},
      params : {},
      sendFileToServer : true //If you want to upload file to your server along with s3
}

Hope this helps you.

harishankards · 2018-05-14T10:08:04Z

Hi @vrajroham,

Thanks for the response.

I tried the same way as you mentioned:

dropzoneOptions: {
          url: 'http://localhost:3000/attachments',
          thumbnailWidth: 150,
          maxFilesize: 0.5,
          headers: { 'Authorization': 'Bearer ' + this.token },
          addRemoveLinks: true,
          dictDefaultMessage: "<i class='fa fa-cloud-upload'></i>UPLOAD ME"
        },
        awss3: {
          signingURL: 'http://localhost:3000/attachments', // Where you will get signed url
          headers: { 'Authorization': 'Bearer ' + this.token },
          params: {},
          sendFileToServer: false // If you want to upload file to your server along with s3
        }

and the my backend server sent this as response:

{ method: 'PUT',
  key: 'images/my_image.jpg',
  url: 'https://student-burger.s3.amazonaws.com/images/my_image.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAITZKxxxxHPB7M4UA%2F20180514%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20180xxx14T100307Z&X-Amz-Expires=86400&X-Amz-Signature=e07717679ee7xxxxxxxx6b366a9812a53e635f5&X-Amz-SignedHeaders=host' }

But, nothing happened in the frontend. So, I checked the console, it is displaying this error

Uncaught TypeError: Cannot convert undefined or null to object

at vue2Dropzone.js?2af3:1.

Also, I want to send the filename to my backend for signing URL. How do I get those?

vrajroham · 2018-05-14T10:24:32Z

@harishankards I got it. I think you should read the documentation completly.

Uncaught TypeError: Cannot convert undefined or null to object

Your response is in invalid format. Your endpoint should return signature object exactly as mentioned in docs. Currently, it's returning all values appended to URL.

Your response should be like as below, More info here

{
   "signature":{
      "Content-Type":"",
      "acl":"public-read-write",
      "success_action_status":"201",
      "policy":"abc123",
      "X-amz-credential":"AKIAIM3WELV3PLALOYDQ\/20171012\/us-west-2\/s3\/aws4_request",
      "X-amz-algorithm":"AWS4-HMAC-SHA256",
      "X-amz-date":"20171012T054729Z",
      "X-amz-signature":"5227d84360d92ef8al45549805b3746f2f1d6641df8986aamcr939c35513cd7c",
      "key":"images/my_image.jpg"
   },
   "postEndpoint":"\/\/student-burger.s3.amazonaws.com\/images"
}

Also, I want to send the filename to my backend for signing URL. How do I get those?

Already note about this is provided in docs.

Firstly, your file is uploaded to S3 then request is sent to your server with file as expected behaviour. But extra field to your request will be added as s3ObjectLocation containing location of your S3 object/file, which you may require to store in database.

harishankards · 2018-05-14T13:07:47Z

Hi @vrajroham,

Got it. Thanks for the explanation. Now I'm able to solve the previous issue, and got up with another one now.

{
"signature": {
"Content-Type":"",
"acl":"public-read-write",
"success_action_status":"201",
"policy":"Policy1xxxxx45161391",
"X-Amz-Algorithm":"AWS4-HMAC-SHA256",
"X-Amz-Credential":"AKxxTZKxxxxxxB7M4UA%2F20180514%2Fus-east-1%2Fs3%2Faws4_request",
"X-Amz-Date":"20180514T130047Z",
"X-Amz-Expires":"86400",
"X-Amz-Signature":"25cadc433cxxxxxxxxbdde29e0ea8ec150134bba753b1e568822696094a8c82c58",
"X-Amz-SignedHeaders":"host",
"key":""
},
"postEndPoint":"//student-burger.s3.amazonaws.com/images"
}

The above is the one I'm sending from backend. Now, the error I'm getting is:
POST http://localhost:8080/student/projects/undefined 404 (Not Found) at dropzone.js?a0b9:2777.

Not sure, why it is coming. But this error where I'm receiving in the page has the route http://localhost:8080/student/projects/new.

vrajroham · 2018-05-15T06:06:28Z

What is your url in dropzoneOptions?
Are your files getting uploaded to S3? (Check network tab)

harishankards · 2018-05-15T08:26:38Z

@vrajroham ,

This is my dropzoneOptions :

dropzoneOptions: {
          url: 'http://localhost:3000/attachments',
          thumbnailWidth: 150,
          maxFilesize: 0.5,
          headers: { 'Authorization': 'Bearer ' + this.token },
          addRemoveLinks: true,
          dictDefaultMessage: "<i class='fa fa-cloud-upload'></i>UPLOAD ME"
        }

Nope. My files are not getting uploaded to S3. No sign of that post/put request to S3 in networks tab.

vrajroham · 2018-05-15T09:10:26Z

Share your complete component and js code.

abdulbasith7145 · 2018-05-15T09:28:19Z

@vrajroham
I'm encountering an issue similar to this, can you please check this issue #328

vrajroham · 2018-05-15T11:01:44Z

@abdulbasith7145 Will check it.

harishankards · 2018-05-15T11:10:12Z

Sure @vrajroham, Please have a look below.

<template>
  <vue-dropzone ref="myVueDropzone" :destroyDropzone="false" 
    @vdropzone-success="vsuccess" @vdropzone-removed-file="vremoved"
    id="dropzone" :options="dropzoneOptions"
    :awss3="awss3"
    v-on:vdropzone-s3-upload-error="s3UploadError"
    v-on:vdropzone-s3-upload-success="s3UploadSuccess">
  </vue-dropzone>
</template>

<script>
  import vue2Dropzone from 'vue2-dropzone'
  import 'vue2-dropzone/dist/vue2Dropzone.css'
  import { eventBus } from '../../main.js'

  export default {
    name: 'upload',
    components: {
      vueDropzone: vue2Dropzone
    },
    data: function () {
      return {
        token: this.$ls.get('token'),
        dropzoneOptions: {
          url: 'http://localhost:3000/attachments',
          thumbnailWidth: 150,
          maxFilesize: 0.5,
          headers: { 'Authorization': 'Bearer ' + this.token },
          addRemoveLinks: true,
          dictDefaultMessage: "<i class='fa fa-cloud-upload'></i>UPLOAD ME"
        },
        awss3: {
          signingURL: 'http://localhost:3000/attachments/signedUrlPut', // Where you will get signed url
          headers: { 'Authorization': 'Bearer ' + this.token },
          params: {},
          sendFileToServer: false 
        }
      }
    },
    methods: {
      vsuccess (file, response) {
        this.$store.state.projectUploadedFile.push({
          filename: file.name,
          signedUrl: response.url
        })
        eventBus.$emit('uploadedFile', {
          projectUploadedFiles: this.$store.state.projectUploadedFile
        })
      },
      vremoved (file, error, xhr) {
        const filename = file.name
        const signedUrl = this.$store.state.projectUploadedFile.map(item => {
          if (item.filename === filename) {
            const index = this.$store.state.projectUploadedFile.indexOf(item)
            this.$store.state.projectUploadedFile.splice(index, 1)
            return item.signedUrl
          }
        })
        this.$http({
          method: 'delete',
          url: '/attachments',
          data: {
            filepath: filepath
          },
          headers: {
            'Authorization': 'Bearer ' + this.token
          }
        })
        .then(function (attachmentDeleted) {
          console.log('attachment deleted', attachmentDeleted)
        })
        .catch(function (attachmentDeleteErr) {
          console.log('unable to delete attachment', attachmentDeleteErr)
        })
      },
      s3UploadError () {
        console.log('s3UploadError')
      },
      s3UploadSuccess () {
        console.log('s3UploadSuccess')
      }
    },
    created () {
      this.$store.state.projectUploadedFile = []
    }
  }
</script>

So, this is my component and I'm importing this component to the other components wherever needed.

vrajroham · 2018-05-15T11:51:09Z

@harishankards Just to test, please do following steps and provide the outcome.

Visit https://rowanwins.github.io/vue-dropzone/docs/dist/#/aws-s3-upload
Select any image/file.
Enter your signing URL http://localhost:3000/attachments/signedUrlPut in textbox (Whitelist this endpoint so, Authorization token will not be required for time being)
Monitor your console and network tab and click upload button.

Let me know the outcome.

harishankards · 2018-05-15T13:34:57Z

Hi @vrajroham,
I got two errors in the console itself.

POST https://rowanwins.github.io/vue-dropzone/docs/dist/undefined 502 () at urlsigner.js:82
Network Error : Could not send request to AWS. (Maybe CORS error) at bundle.3fb36dea395ce2168db7.js:1805
I doubt why CORS is happening as it has the signed URL with it.

harishankards · 2018-05-17T23:22:49Z

Hi @vrajroham,

Just wanted to check whether do I have got any updates.

Looking forward

WajdiF · 2018-05-19T19:29:18Z

Hi @vrajroham
I have the exact same problem, could you please help ?
Best regards

harishankards · 2018-05-21T12:31:22Z

Welcome to the club @WajdiF. If you have got some solution, please let me know.

Seems @vrajroham is busy

rowanwins · 2018-05-21T13:26:01Z

Hi @harishankards & @WajdiF

Can either of you provide a more detailed error message?

For your s3UploadError method you should be able to pass in the actual error message

      s3UploadError (e) {
        console.log(e)
      },

And also just a friendly reminder this is an open-source project - we do our best to support users but we do ask for a bit of patience as both @vrajroham & myself have families to care for, full-time jobs, and other various life responsibilities. We maintain this component in our spare time with no financial gain. Anyone is free to dig into the source code to work out what is going on.

Thanks,
Rowan

vrajroham · 2018-05-21T19:12:56Z

@harishankards @WajdiF @abdulbasith7145

I spent much time to replicate your issue, but I'cant! Maybe I need more information, which can be provided as Rowan suggested in the previous comment. I have recorded sample video where, you can see the flow how I'm uploading an image to s3, from this doc page (testing purpose).

Note: In the video, I used ngrok tool to expose my (url signer endpoint) local endpoint publicly due to pre-flight error in js.

It's really very hard to take out time from daily schedule, though we are trying our best. What I suggest to you guys is, any of you can look into the code and find out where it is going wrong. It is really simple to start with, like below.

Clone the repo
npm install / yarn install
npm run start / yarn start

If anyone can provide more specific error then it will be really helpful.

Thanks.

harishankards · 2018-05-22T07:36:33Z

Hey @rowanwins,

Thanks for the detailed explanation and yes I understand bro. Regarding the solution, for me, it doesn't go until s3UploadError(e){} and all.

Whenever I upload something, in the console it throws: http://localhost:8080/student/projects/undefined 404 (Not Found) at dropzone.js?a0b9:2777 but I don't know how it is taking that URL.

And @vrajroham, I clearly understand and thanks a lot for the video. I also used ngrok and I tried with this page, but this time it throws this error:
POST https://rowanwins.github.io/vue-dropzone/docs/dist/undefined 502() at urlsigner.js:82.

So, my find is the request is made to whatever the page's current url + undefined. Because when I tried with mine it was http://localhost:8080/student/projects/undefined and with your docs page it is POST https://rowanwins.github.io/vue-dropzone/docs/dist/undefined.

So, something fishy is happening there only.

Note: I also understand that contribution is needed to the repo. But I'm currently locked down until this month end. I shall contribute once my sprint is over.

WajdiF · 2018-05-23T12:30:35Z

Hi, @rowanwins,@vrajroham

Thanks a lot for your help and for this library, i totally understand your lack of time. I will do my best to help solve this issue (if any :) ).

I cloned the lib and tested locally, i found a weird behavior :
urlSigner.js line 56, when we try to get the response.postEndPoint the result is undefined. this is the root cause of the issue.
The wierd thing is postEndPoint attribute is defined but accessing it with '.' returns an undefined but accessing with braces returns the correct values, hereunder some logs (after cleaning signature and private data) :

{postEndPoint: "//s3-eu-west-3.amazonaws.com/myBucket", signature: {…}}
response.hasOwnProperty('postEndpoint') => false
[] access, response['postEndPoint'] => //s3-eu-west-3.amazonaws.com/myBucket

the following exception is thrown when the postEndPoint is undefined :
urlsigner.js?77a0:87 POST http://localhost:8081/undefined 404 (Not Found)

do you

rowanwins · 2018-05-23T12:38:25Z

Gday @WajdiF

Great digging! I wonder if this is a browser compatibility issue, are you using IE by any chance, if so perhaps #325 is related?

horaciosolorio · 2018-06-28T16:50:28Z

@harishankards

Hi can you help me please i using node but y only have URL from S3 AWS SDK in back end but i dont understand how to get in node return "signature" object.

Below is the current code in node i have

const parametros = {
Bucket: myBucket,
Key: myKey,
Expires : signedUrlExpireSeconds,
ACL:'public-read',
ContentType:'image/png'
}

var url = s3.getSignedUrl('putObject', parametros , function (err, url) {
if (err) {
res.send(err)
} else {
console.log(url)
res.send(url)
}
})

Please help me

ShrJoshi28 · 2019-01-31T06:42:05Z

how to solve this issue-
ccess to XMLHttpRequest at 'https://4kd2hi3ere.execute-api.us-east-1.amazonaws.com/default/myfirstvuecode' from origin 'http://localhost:8080' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.

ianwow · 2019-10-03T04:59:06Z

@ShrJoshi28 In the past I've gotten a CORS error because I didn't add a CORS policy to my S3 bucket. I've also gotten that error because I was not using a region-specific URL for my upload to S3. In that case my browser said I had a CORS error even though it was not actually a CORS error but rather a redirect error (something about preflight request couldn't be redirected). If you look closely at the redirect you'll see the browser was redirected to a region-specific URL.

If you're not using the us-east-1 (Virginia) region, you need to upload to a region-specific URL. For more info, see "region-specific endpoints" at https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html

I created a pull request to ensure Dropzone does this correctly:
#494

vrajroham added the help wanted label May 14, 2018

vrajroham added the AWS S3 label May 15, 2018

vrajroham mentioned this issue May 21, 2018

"Direct upload to s3" description is not sufficient. #328

Open

vrajroham self-assigned this Feb 19, 2019

vrajroham added the Important label Feb 19, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How to upload to S3 after getting the signed Url from the backend? #327

How to upload to S3 after getting the signed Url from the backend? #327

harishankards commented May 13, 2018

vrajroham commented May 14, 2018

harishankards commented May 14, 2018

vrajroham commented May 14, 2018

harishankards commented May 14, 2018

vrajroham commented May 15, 2018

harishankards commented May 15, 2018

vrajroham commented May 15, 2018

abdulbasith7145 commented May 15, 2018

vrajroham commented May 15, 2018

harishankards commented May 15, 2018 •

edited

Loading

vrajroham commented May 15, 2018 •

edited

Loading

harishankards commented May 15, 2018

harishankards commented May 17, 2018

WajdiF commented May 19, 2018

harishankards commented May 21, 2018

rowanwins commented May 21, 2018

vrajroham commented May 21, 2018 •

edited

Loading

harishankards commented May 22, 2018

WajdiF commented May 23, 2018 •

edited

Loading

rowanwins commented May 23, 2018

horaciosolorio commented Jun 28, 2018

ShrJoshi28 commented Jan 31, 2019

ianwow commented Oct 3, 2019

How to upload to S3 after getting the signed Url from the backend? #327

How to upload to S3 after getting the signed Url from the backend? #327

Comments

harishankards commented May 13, 2018

vrajroham commented May 14, 2018

harishankards commented May 14, 2018

vrajroham commented May 14, 2018

harishankards commented May 14, 2018

vrajroham commented May 15, 2018

harishankards commented May 15, 2018

vrajroham commented May 15, 2018

abdulbasith7145 commented May 15, 2018

vrajroham commented May 15, 2018

harishankards commented May 15, 2018 • edited Loading

vrajroham commented May 15, 2018 • edited Loading

harishankards commented May 15, 2018

harishankards commented May 17, 2018

WajdiF commented May 19, 2018

harishankards commented May 21, 2018

rowanwins commented May 21, 2018

vrajroham commented May 21, 2018 • edited Loading

harishankards commented May 22, 2018

WajdiF commented May 23, 2018 • edited Loading

rowanwins commented May 23, 2018

horaciosolorio commented Jun 28, 2018

ShrJoshi28 commented Jan 31, 2019

ianwow commented Oct 3, 2019

harishankards commented May 15, 2018 •

edited

Loading

vrajroham commented May 15, 2018 •

edited

Loading

vrajroham commented May 21, 2018 •

edited

Loading

WajdiF commented May 23, 2018 •

edited

Loading