Signed URL — secure files in AWS S3 bucket

Well, sh*t happens!

After 100500 lines of code and sleepless nights your app is ready, deployed and is tested by the first users. Hell yeah!.. Seems like you can chill a bit, but now is good time to think about security.

Let's start securing our images and other files.

Login to your AWS account and select S3 option. Find bucket you want to secure, in our case it is test-bucket-images1

As we see all files have public access and you can easily open link from any origin, just tap on Object Url of any file.

To fix that select go back to the bucket and select Permissions tab, click edit next to Block public access button. To secure your bucket choose next 2 options and save changes:

• Block public access to buckets and objects granted through new access control lists (ACLs)
• Block public access to buckets and objects granted through any access control lists (ACLs)

Next step is to modify bucket policy.

We will add 2 policies for now, in future you may add much more options. Here you can read a bit more about policies and options available

In resource we have to specify Amazon resource name (ARN), you can find it in Properties tab — Bucket Overview.

1. First policy will restrict access to resources via HTTP (allow only secure HTTPS);
2. Second policy will restrict access outside of VPC. Note that next to aws:sourceVpc we indicate VPC id. You can find it in VPC section (type it in search). It always starts from 'vpc-…..'

Save changes and try to open file url in browser — you will get error Access denied. That's exactly what we wanted

Now we can check if we have access inside our VPC.

Go to EC2 from the main menu > Click on Instances > select Instance which needs access and tap on Connect button. Generate SSH key and follow detail instructions how to connect to instance.

Once connected type in terminal aws configure. Hit enter to skip 1st 2 options, default region name — Indicate your region.

Then type in terminal:

cd ~/.aws

aws configure

Enter AWS Access key id, Secret access key and default region to register.

nano credentials (check config file, credentials should match)

Now lets create text file and try to upload it to bucket from cli

echo “123456789abcdefg” > textfile

aws s3api put-object — key text01 — body textfile — profile yourUserName— bucket test-bucket-images1

The file will be uploaded to bucket, you can check it in the aws account.

We have achieved that our files are not available outside VPC. However what happens if we have to display these picture on the website or in our app?

There are 2 options:

1. Edit policy rules and add referrer condition which will indicate to origin of the request.

{
 “Sid”: “Allow get requests originating from *.facebook.com”,
“Effect”: “Allow”,
 “Principal”: “*”,
 “Action”: “s3:GetObject”,
“Resource”: “arn:aws:s3:::test-bucket-images1/*”,
“Condition”: {
   “StringLike”: {
      “aws:Referer”: “https://*.facebook.com/*”
    }
  }
}

Now images will be loaded on all resources where domain name ends with facebook.com. However this option is not very secure, as it can be any random similar domain *ksjdjgsdcjhgsdjhcg.facebook.com/ for e.g

Moreover user will be able to save link or send it somebody, so this option is not advices.

2. Create Signed URL with CloudFront

1. Firstly lets generate public and private keys locally on our machine. Open separate terminal window:

openssl genrsa -out private_key.pem 2048

openssl rsa -pubout -in private_key.pem -out public_key.pe

Here you can check guide from aws, which can give a bit more info

2. Now we have to attach keys to Go to CloudFront (type it in search) > Public keys > Add public key button

In modal enter key name, comment and key value.

To get key value type we need to open public_key.pem file and copy value which is inside, or just type in terminal to display value:

cat public_key.pem

3. Next step is to add key to the group, go back to cloudFront > Select Key groups option from sidebar menu .

Select our key from dropdown and hit add button.

Public key ID has been generated, we will need it a bit later

4. Now we will have the largest part, go back to CloudFront > Distribution section on the left menu > Create Distribution button > Get started.

in the Origin Settings section (Origin domain name field), we will select an Amazon S3 bucket that we want to make available for users via signed url. It has to be resource name like arn:aws:s3:::test-bucket-images1, you can find exact bucket resource name if go back to s3 > select your bucket > Properties > Amazon resource name (ARN)

Restrict bucket access — YES.

Grand read permissions on bucket — YES.

If you do not have separate folders in your bucket leave origin path blank.

Default cache behavior — most options are default, however very important to change:

Viewer protocol policy — Redirect HTTP to HTTPS

Restrict Viewer Access — YES, so user will have access to file links only via signed url

Trusted Key Groups or Trusted Signer — Trusted key groups

Trusted key groups — select group in the list we have created before and click ADD button

Distribution created, you need to wait a few minutes for deploy.

5. Final step — we need to create signed URL

Here you can find examples for different languages

I was using nodejs and aws-cloudfront-sign package

Add package to your BE dependency list

npm install aws-cloudfront-sign

Copy private key value (cat private_key.pem)

const privateKeyString =
'-----BEGIN RSA PRIVATE KEY-----\n'
'wkjewdnUYGUHV8347rsdbKJJJJBxbskxjbBt3A7Z4aBFAUvLiITzmHRc4UPwryJp\n'
'asskjd7rsjdhbc48838sjdbsdk,kKJjbh8snkJJHGSJKSLS:SLNjdhchI/09+j1h\n'
'tuf/sadckadfknvbajehfbjaehbfjhadsbfjhbasdjch847e7xmkVV0KewO02wJz\n'
'jBfDw9B5ghxKP95t7/B2AgRUMj+r47zErFwo3OKW0egDUpV+eoNSBylXPXXYKvsL\n'
'Asdckjnkj&&jahdbf8s7dhcbduhfcvsadkjcnkjnsef894393484mjtHL7P9/z4B\n'
'KdODlpb5Vx+54+Fa19vpgXEtHgfAgGW9DjlZMtl4wYTqyGAoa+SLuehjAQsxT8M1\n'
'Bskdjcnksdajvnbjahdbfvisd8dsicuhsadc87ydvygadfjhdkafnsdajcsc74k=\n'
'-----END RSA PRIVATE KEY-----'

Then create function which will generate signed URL.

const cf = require('aws-cloudfront-sign')
const options = {
  keypairId: 'K20CI4W08D7Y0W', 
  privateKeyString: privateKeyString,
//timestamp
  expireTime: new Date().getTime() + 24*60*60*10000,
}
const signedUrl = cf.getSignedUrl(
  'http://xxxxxxx.cloudfront.net/picture.jpg',
   options
);
console.log('Signed URL: ' + signedUrl);

KeypairId can be found in CloudFront > Key Management option from the left sidebar > Public keys. Next to each key there is ID, we need the one created a few steps ago.

expireTime — time, when our link expire. Has to be timestamp > now.

Function cf.getSignedUrl has 2 params:

1. Original link to our file after distribution. It has 2 parts:

Domain name of the distribution, you can find it next to distribution id

Original link to our file, you can find it when go to the bucket

2. options

After we launch function our signed link will look like:

https://dsdclkmsmsl.cloudfront.net/image.jpg?Expires=16646464441&Policy=eksdcnkasdjcnkasdcnsdcjsdcjM6Ly9kM3Bob3o3cDhjY2xkbC5jbG91ZGZyb250Lm5ldC8yMDIwMDkyMsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE2MTQ4MTg3MjF9fX1dfQ__&Signature=wsdcknsdcjdnsaaWjQzyX4cRIuW1sdfbdjfbhjasdfbhd777dhdsbq3ymH4tbfNVtQ~o2iCZkZ8u5mXtd9NWz1Tn05De20CfF-FwemlKYnBB3hnu1zw__&Key-Pair-Id=KHHV66hH21T99

Now you can easily access it from the browser