Amazon S3 operations using signed url's

Amazon S3 is one of the most popular services in AWS suite and has SDK’s available for almost all popular languages and frameworks. One core aspect of our microservice architecture at Shippable is that all services interact with a common API service which serves as the only point of contact for CRUD operations and permission resolution for all objects in the system. One important function of API is to provide credentials to any requesting microservice e.g. if a microservice wants to connect to S3 bucket, then API is required to provide the required credentials to do that. One of the secure ways of doing this is to provide any requesting service with temporary signed URL’s. AWS SDK comes with this functionality and its simple enough to implement. This can be done in two simple steps. I’ll use AWS Javascript SDK for this example.

tl;dr the code for doing all the steps is available HERE

1. Create signed URLs

The getSignedUrl function http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#getSignedUrl-property provides an easy way to create signed urls for supported S3 functions

  var options = {
    Bucket: BUCKET,
    Key: OBJECT_PATH,
    Expires: EXPIRATION_TIME_SECS
  };

  params.s3.getSignedUrl('putObject', options, function(err, url) {
    // the 'url' is a short lived url that
    // expires in EXPIRATION_TIME_SECS,
    // only supports PUT on the object at OBJECT_PATH
    // in the bucket BUCKET
  });

replacing putObject with getObject and deleteObject will give short lived URLs for respective operations. Running the gist provided above will generate an output similar to following

<--- some log lines --->
12 Jan 16:44:39 - Accessors : { get: 'https://s3.amazonaws.com/myTestBucket/path/where/the/object/is/store?AWSAccessKeyId=FOO&Expires=1484271879&Signature=BAR%3D',
  put: 'https://s3.amazonaws.com/myTestBucket/path/where/the/object/is/store?AWSAccessKeyId=FOO1&Expires=1484271879&Signature=BAR1%2FOH0Ezc%3D',
  delete: 'https://s3.amazonaws.com/myTestBucket/path/where/the/object/is/store?AWSAccessKeyId=FOO2&Expires=1484271879&Signature=BAR2%3D' }
12 Jan 16:44:39 - Done

2. CURL’ing the URLs

To save a file on S3 bucket BUCKET at path OBJECT_PATH using curl, simply do the following

$ curl -v -XPUT '<url in the PUT section of accessors>' -H 'Content-type: <whatever is file content type>' -T '<file name>'

To delete the file, do following

$ curl -v -XDELETE '<url in the DELETE section of accessors>'

and to get the file

$ curl -v -XGET '<url in the GET section of accessors>'

Pretty straightforward, isnt it !!!

Share Comments
comments powered by Disqus