Image Upload on AWS S3 using API Gateway and Lambda in Python

AWS Services: AWS API Gateway, AWS Lambda, AWS S3

Getting started on AWS Services can be a bit daunting. The docs seem so expansive and we are impatient to read each and every detail. For a beginner, the basic functionality can be set up in a few simple steps. And then we go from there to the deeper configurations.

So here I am going to write about the basics of AWS Lambda and how to trigger it with an API Gateway so that an image can be uploaded on an S3 Bucket.

We can upload an image directly on S3 Bucket by using S3 functions from the client end. It is detailed on my other blog post here. But here we are going to take the help of an API to do it.

Sign up to your AWS account and go into the console

You need to sign up to an AWS account to be able to use the AWS services. The sign up process is pretty simple and they provide most of the services for free for a full year. After that, it is billable according to the extent of the use of those services.

Create an AWS API Gateway

Go into the console and search for API Gateway. A window will open and there will be a ‘Create API’ button on it. Click on it.

Fig. 1 Creating an API in API Gateway Console

Here you will select the option to create a REST API and enter the name of your API. Then click on ‘Create API’.

We need to create a new resource to configure our POST request. Click on Actions -> Create Resource and a new form will open up. I am naming this resource as ‘upload-to-s3’.

Don’t forget to check on ‘Enable API Gateway CORS’ as without it there are going to be cross-origin errors.

Fig 2. Create a resource for an API

Once the resource is created we need to define the call we would like to make. Click on Actions -> Create Method.

Select POST from the drop-down and click the correct option.

Fig 3. Creating POST Method for the API

A new window will open up where we have to set up the configurations. Set the integration type as Lambda Function and tick the checkbox of Use Lambda Proxy Integration.

Type the name of the Lambda Function that you are about to create and click on ‘Save’.

Your API Gateway is now configured! Now we will have to create the Lambda Function.

Create a Lambda Function

Go to the console and type ‘Lambda’

A new window will open up which will list all your existing Lambda Function.

Click on ‘Create Function’ to create a new Lambda Function. A new window will open up.

Fig 4. Creating a new Lambda Function

We don’t have an existing IAM role for our Lambda Function. So we have to create one so that our Lambda Functions can access other services as well.

Go to IAM Console from the search console. Click on ‘Roles’ and then click on ‘Create Roles’

Fig 5. Creating a new role in IAM Console

Choose Lambda as the option to use the service.

Choose the AmazonS3FullAccess and CloudWatchLogsFullAccess policy and press Next.

You can add Tags but they are optional. Click ‘Next’.

Fig 6. Review the role

Type in your role name and review your role and click on ‘Create Role’

Now your role has been created so we can go back to create our Lambda Function!
Go back to the Lambda Console and click on ‘Create Function’

Fig 7. Finally creating a new Lambda Function

Take the Lambda function the same as the one we are using in the API Gateway. We choose the Role as the one that we have just created.

Here we will be choosing the run-time environment as Python 2.7.

Now click on ‘Create Function’ to create the function!

A new window will open where we connect the two services with the Lambda Function.

Fig 8. Choosing services to connect to AWS Lambda

Choose ‘API Gateway’ from the left sidebar as you want to trigger the Lambda Function from an API call.
As we have already configured the IAM role so AWS S3 will be automatically connected to the Lambda function.

We have not deployed our API yet so we will go back to the API Gateway and to our API to deploy our API.

Go to the API that we created, select on ACTIONS -> Deploy API

We will name the stage as ‘prod’ and then click on deploy.

After that, you will get an invoke URL of your API through which you will have to make a call to. It’ll look something like this.

https://<invoke url>.amazonaws.com/prod/upload-on-s3

Now, we go back to the Lambda function console and connect the API Gateway to it.

Fig 9. Configuring API Gateway triggers to invoke the Lambda Function

Select the name of our API and set the stage name that we just created and click on ADD and click on Save.

Now our Lambda Function will be triggered with an API Call!

You can test it by calling the API in the Test Console in the API Gateway. It will return a response from the Lambda Function.

Creating a S3 Bucket

Now we need to create a bucket so that we can upload our images on it.

Go to the search console and select ‘S3’ on it.

The console will open and click on ‘Create Bucket’. A wizard will open up and you have to fill in all the details as shown.

Fig 10. Creating and configuring a S3 Bucket

Go through the configuration and set it up according to your requirements.

Now that the bucket is successfully created, we can write the Python script for the Lambda Function!

import json
import boto3
import base64
s3 = boto3.client('s3')
def lambda_handler(event, context):
    print event
    if event['httpMethod'] == 'POST' : 
        print event['body']
        data = json.loads(event['body'])
        name = data['name']
        image = data['file']
        image = image[image.find(",")+1:]
        dec = base64.b64decode(image + "===")
        s3.put_object(Bucket='upload-to-s3-project', Key=name, Body=dec)
        return {'statusCode': 200, 'body': json.dumps({'message': 'successful lambda function call'}), 'headers': {'Access-Control-Allow-Origin': '*'}}

Here we are using boto3 to access s3. Boto is the Amazon Web Services (AWS) SDK for Python, which allows Python developers to write software that makes use of Amazon services.

We are taking a base64 image from the client side and decoding it back into an image through the Lambda Function.

After decoding, we call the boto3 s3 upload function to upload the image on S3 with the required parameters.

You can use CloudWatch logs after an API call to see what is being received in the Lambda Function from the client side and if there is an error, it’ll help to to debug it.

Now the work is done.

When we invoke the POST Request and send a JSON file through the API containing the base64 code of the image and the images’s name, it triggers the Lambda Function which then uploads the file in the S3 Bucket.

We have learnt how to connect several AWS Sevices to connect with Lambda and trigger it to perform a task. Have fun experimenting with the AWS Services!