Work with S3 Object Lambda in Typescript

Recently AWS announced S3 Object Lambda, which enables data processing as it is being retrieved from S3. By using S3 Object Lambda, you can easily present multiple views from the same dataset, and you can update the Lambda functions to modify these views at any time.

For example, we have a user dataset stored in S3, the dataset has user email, name, and address. When the data is requested by an email marketing application via an S3 Access point, we don’t want to expose the address data, whereas when requested by a post marketing application we don’t want to expose the email data. Let’s write some code in Typescript to do it.

Lambda

The lambda function is simple, it retrieves the object from S3, modifies it, and then overwrites the response.

There are a couple of key points.

1. event.configuration.payload This is the value we created in the Object Lambda Access Point configuration, you can use it for the purpose of adding extra business logic conditions because you can have multiple access points links to the same Lambda function.
2. writeGetObjectResponse It basically overwrites the response to the GetObject request, just be careful with the type of response body, you may need some data conversion when you set the response body. In my example, I need to convert to String because modifiedData is a JavaScript object.
3. IAM Role permission. You need at least two Policies. The first one is the managed AWSLambdaBasicExecutionRole. The second one is to give the function permission to invoke writeGetObjectResponse

Object Lambda Access Points

It is very simple to create an Object Lambda Access Point, it simply connects the lambda function we created above to the S3 Access Point.

NOTE. Once the Object Lambda Access Points, you should use the Object Lambda Access Points ARN to access the data in order to trigger the object lambda, not the original Access Point ARN. It looks like

arn:aws:s3-object-lambda:ap-southeast-2:123456789:accesspoint/s2k

Test

There are many ways to test it’s working, the simplest is by using AWS CLI.

aws s3api get-object --bucket arn:aws:s3-object-lambda:ap-southeast-2:123456789:accesspoint/s2k --key data.json outfile

Just be aware that your AWS CLI is the latest, otherwise, you will see this error, this is because the old version doesn’t support Object Lambda Access Point ARN.

Or if you like you the script does the same job well.

Amazon S3 Object Lambda is still very young, there are many use cases that will benefit from this new serverless weapon. At the same time, there are not many examples out there. If you have any questions, feel free to reach out!

As always the demo code is at https://github.com/crespowang/s3-object-lambda/tree/master. Happy Clouding!