How to use AWS (Amazon Web Services) S3 Bucket through a Rails App? (Guide for beginners)[Tech]

Trust me, I panic a lot when I need to implement something new.
If you ask my colleagues, they will tell you something like this:

OMG! You panic a lot! Try to focus on what needs to be done!

Don’t overthink — just take one step at a time!

One such time when I panicked but later, with guidance, accomplished to explore and work on was leveraging an AWS S3 bucket through a Rails app, which I have learned through research and guidance from mentors and is compiled here in a few easy steps.

This blog is intended for beginners who, like me, are attempting to solve a simple problem statement of pushing and pulling files from and from an AWS S3 bucket.

So as advised, let's proceed step by step:

Step 1: AWS support for Rails:

AWS offers an AWS SDK for Ruby, which allows users to do some typical Amazon S3 operations. There is an official gem called aws-sdk-rails that can be used to use this SDK (refer to the documentation for initial setup-related guidance). To use the gem, add the following lines to Gemfile:

gem 'aws-sdk-rails', '~> 3'
gem 'aws-sdk-s3', '~> 1'

Or to leverage the SDK (access the classes and methods), add the below line to the top of the file:

require 'aws-sdk'

Step 2: AWS Resource configurations
Next, one needs to setup the resource with the specific configurations — (basic code snippet below)
Here, the S3 object will have the S3 resource object for the “us-east-1” region.

s3 = Aws::S3::Resource.new(
  region: 'us-east-1',
  access_key_id: '...',
  secret_access_key: '...'
)

An alternative and more safe method is to use the environment variables and setup the AWS config as shown below:

Aws.config.update({
 credentials: Aws::Credentials.new(
  ENV['S3_ACCESS_KEY_ID'],
  ENV['S3_SECRET_ACCESS_KEY']
 ),
 region: ENV['AWS_REGION']
})

s3_resource = Aws::S3::Resource.new

Step 3: Access the S3 Bucket

Using the above-mentioned s3_resource, access the S3 bucket:

s3_bucket = s3_resource.bucket(ENV['S3_BUCKET_NAME'])

Step 4: Basic Operations

Push file to S3 bucket
One can push files to the S3 bucket using the upload_file method of the SDK.
First, create the S3 bucket where the file needs to be stored by providing the destination path.
Theupload_file method here takes two inputs: first, the file itself (accessed using the name of the file here), and second, the acl (access control list) (specifying what type of access needs to be set to the file, here "public read" so that anyone having the public url can access the file). Read more about ACL here.

file_name = File.basename 'file_to_be_uploaded'
path_on_s3_where_file_be_stored = 'destination/path/'
object = s3_bucket.object(path_on_s3_where_file_be_stored)
object.upload_file(file_name, acl: 'public-read')
public_url = object.public_url

Check if the file is present in the S3 bucket or not.

Reference: Check if a file exists on AWS S3 path using aws-sdk gem

The easy way would be to try to get the file. Either the object returns with the file or is nil. But to be specific, if you want to check any other meta data, use the head_object function.

head_objectreturns a struct with relevant meta data about the file if it is present on the path.

head = s3_resource.head_object(
 bucket: s3_bucket,
 key: path_on_which_file_is_expected
)

--response
#<struct Aws::S3:: Aws::S3:: Types::HeadObjectOutput meta-data here..>

If it is not present, then an Aws::S3::Errors::NotFound is raised. So one can create a simple method to check if a file is present or not and then proceed.

def s3_exists?(bucket, path)
  s3.head_object(bucket: bucket, key: path)
  true 
rescue Aws::S3::Errors::NotFound => e
  false
end

Pull file from S3 bucket
Get the files from a specific path (after checking if it exists or not) to a response target path using theget method of the SDK. The get method takes in a response_target input — the path where the file needs to be downloaded (location on the system where the Rails App / ruby script is running)

path_on_s3_where_file_is_stored = 'destination/path/'
obj = S3_BUCKET.object(path_on_s3_where_file_is_stored)
download_path = 'path_where_the_file_needs_to_be_downloaded_to'
obj.get(response_target: download_path)

The file will be available for access at the response target.

Individually or by zipping and then pushing the zip folder itself, the aforementioned operations can be carried out on any type of file—images, videos, graphics, audios etc.

I hope this blog gives all beginners a quick overview of how simple it is to implement AWS S3-related operations from a Rails app.

Will be coming up with other such intersting yet easy topics which made me panic! 😅

- V.G

PS: Special credit goes to my mentors and colleagues, Anuja Ware, Pragati Wagh, and Preeti Dantkale, for helping me understand the topic and implement the solution! Thanks to Niranjan Patil for helping me curate it.