Allowing API Gateway to upload objects in S3 directly
In the previous post, we discussed the main features of API Gateway. Let’s explore how to integrate a REST API with S3. Before continuing, make sure to have an S3 bucket for testing purposes. If you don’t have it, create a new one in the same region of your gateway. Keep note of the bucket name. In this demo, “test-s3-apigateway” will be used. Also, you should create an IAM role that allows the gateway to writing on S3. This post will guide you on how to make it.
The first step consists of opening the API Gateway service from your AWS Account and clicking on “build” in the REST API box.
AWS will propose an OpenAPI/Swagger example, but we will start from scratch for this demo. So click on “New API” (1)
Then fill “API Name” (2) with a value of your choice (i.e. “Demo API”), and then clicks on “Create API” (3).
Now we should add a new REST resource to the API. Click on “Actions” (1), then “Create Resource”.
We will create a parent resource called “articles”, which logically act as a “container” in the “New Child Resource” form, and set “articles” as the resource name (1). “Resource path” (2) will be filled automatically. Then click on “Create Resource” (3)
In the next step, we will create a nested resource representing the file we upload on S3. Select “/articles” (1), then click on “Actions” (2), then click on “Create Resource” (3).
A “Key” path parameter is needed to specify the name of the file we are uploading on S3. On the “Resource name” (1), set a handly name, like “object”, and in the resource path (2), set the “{key}” value. The brackets are essential and will mark the key as a parameter you can use to specify the S3 object key. Click on “Create Resource” (3)
After the resource creation, you must add methods (or actions). Typically in a REST resource, a POST method is used to create a resource. Select “/articles/{key}” (1) then “Create method” (2).
On the pop-up menu, select POST, then click on the ✔️sign to confirm.
On this form, we will specify “AWS Service” (1) ad Integration Type, the region of your bucket (2) as AWS region, and “Simple Storage Service (S3)” as AWS Service (3).
When we want to upload an object to S3, we must use the PUT method on the API. We use the same API, so “you must choose PUT” (4) from the HTTP Method select box.
As “Action Type”, select “Use path override” (5), then add your S3 bucket name as Path Override (6), followed by “/articles/{key}”. After completing this tutorial, every time you call this API with a file in the body, API Gateway will upload the file to the specified bucket with the prefix “articles” and the parameter value as the key.
The last value to set is the “execution role” (7) that we have already created (see this post for details).
Click on “Save”.
In the final step, we will map the “key” parameter that we have just set in the “Path Override” option to the “Key” parameter of the S3 PutObject API. So click on the method that we have just created (1), then on “Integration Request” (2).
Open “URL Path Parameters” (1) and click on “Add Path” (2).
On “Name” (1) enter “key”, on mapped from (2) set “context.requestOverride.path.key”. Then click on the confirmation button (3).
The API Gateway configuration is complete, and we can deploy it to a new “stage”.
Click on “Actions” -> “Deploy API” and select a new Stage.
Name the stage with the desired value like “v1”, “demo”, “staging”, or “prod”, then click on Deploy.
After the deployments, we will have the new API stage available. Click on Stages, then click on the name of the stage. We will find an URL that we can use to invoke the APIs.
Copy that URL, in our case, was “https://XXXXXXXXX.execute-api.us-west-2.amazonaws.com/v1”
Pick a full file path of a random file on your computer, and we upload it with the new API. If you are on Mac or Linux, open a terminal and enter:
curl -d @path/to/file/to/upload.txt https://<your-api>.execute-api.us-west-2.amazonaws.com/v1/articles/upload.txtYou will see CURL upload your file! If you open the bucket on the AWS console, you will find the file upload.txt under the “articles/” prefix.
