How AWS Lambda Solved a Major Photo Bug
Triggering multiple lambdas from S3 to rotate images
Residential partners of Vrbo’s™ (part of Expedia Group™) urban solutions product complained that their property images weren’t oriented correctly and they weren’t able to fix them. We solved their problem quickly using AWS Lambda to process images after they are uploaded to S3.
We noticed that the issue was due to browsers not reading the Exif metadata stored on the image. The Exif metadata tells the reader (browsers, phone, etc.) of the image how the image should be rotated so that it is shown correctly to the consumer. Smartphones usually set this metadata when they upload photos. For example, if the picture is taken in landscape orientation and it is uploaded, the picture is technically upside down, but the Exif data tells the reader how to orient it.
We thought that the issue was only in our residential dashboard, so we implemented a way for the browser to read the Exif data and orient the photo correctly. This did solve the issue for the dashboard, but then we discovered that the photos were also displayed incorrectly on the actual Vrbo listings as well! Doing a quick browser fix wasn’t the solution because the actual data for the image needed to change.
A serverless solution
We needed to orient the images correctly prior to storing them and displaying them to the resident or Vrbo travelers. Since we store all of our images in Amazon S3 buckets, we accomplished this by utilizing AWS Lambda functions. With this approach, we can guarantee that the image is stored correctly and that any service (such as the resident dashboard or the resident mobile app) that uploads photos to a bucket won’t have to worry about orienting the photo correctly.
We were already using an AWS Lambda function to create thumbnails for uploads, so we figured adding an additional one for orienting would be the ideal approach.
Our thumbnails are created when a service uploads a photo, which triggers an event that executes a subscribed lambda function. While this process is correct for executing a single lambda function, we needed to execute two Lambda functions in response to a single event, which AWS does not support.
Step one: Trigger multiple lambdas
With lots of Googling and research, we discovered that the way to solve triggering two Lambda functions with one event was to use Amazon Simple Notification Service (SNS). With SNS, we can tell a bucket that when an action happens (uploading a photo in our case), it will trigger a notification that one or more entities are listening to on AWS (i.e., one or more lambda functions). We simply have to tell the bucket to create the notification and tell the lambda function to listen to the notification. When we migrated our CreateThumbnail function over to an SNS trigger, we did have to change the way it parsed the data it was given because the data was coming in as a message (notification) instead of a result (event). The lambda function simply needed to read the data from a further nested download location in the data and parse some new JSON.
We needed to execute two lambda functions in response to a single event, which AWS does not support
Instead of our dev bucket and production bucket triggering an event that directly called the CreateThumbnail lambda function, they were triggering notifications called UploadToTemp (dev) and UploadToUploads (production).
Step two: The new lambda function for orientation
This new lambda function was very similar to the one we used for creating thumbnails because they used the same open-source library: GraphicsMagick (gm) with ImageMagick. When the lambda function is triggered by the SNS notification, the function is provided which bucket the photo was uploaded to and the name of the file. The lambda function downloads the newly uploaded photo and checks its Exif metadata orientation with gm. If the Exif metadata exists, then we simply tell gm to orient the new photo appropriately and overwrite the newly uploaded image with the now correctly oriented one.
Lambda function flow:
> SNS Notification
> Lambda function trigger
> Update image in bucket and log
We try to move quickly on our team, and lambda felt like a perfect fit because we can quickly deploy decoupled, computationally intensive functionality and get it live in days.
If you are interested in triggering AWS Lambda functions from specific events in buckets, I highly recommend setting things up the SNS way right away so that any future Lambda function can easily be brought in and routed correctly. It requires a couple of extra steps, but the payoff is totally worth it. Lambda + SNS is an excellent solution for anyone looking to do computation on S3 bucket data!