Image Resizing Architecture at

Shubham Kukreja
Sep 4, 2018 · 3 min read

The Problem

Images are the core to almost every business on the internet right now be it Hotels, Real Estate, E-commerce etc. With every great image comes not so great image-versions, Search Page has thumb images, Product Page has Medium or Large Images etc. Now consider a scenario you have 100TB of images with a fixed number of image versions and your business demands complete design overhaul of your website. Consider the time and resources you will have to spend to create new versions of images (Approx 6 Cr images). We faced a similar problem a few months back.

Plus Storage cost of 100TB data in S3 (More than the salary of a good software developer in India :))

Lazy Load

What if I tell you that you can generate any version of an image at runtime? The solution that you are thinking is not that complicated!!

Ever Heard of Aws Lambda?. I feel Aws Lambda is the uber of computation industry. No maintenance of servers, no memory leak issues. It just feels like magic. Amazon took it a step further, they introduced LambdaEdge in November last year. Aws LambdaEdge is basically Aws lambda replicated in every edge location (which can be triggered on cloud front request).


The Solution

There are multiple lambdas involved in the solution we built each with a specific function:

Webp checker — This lambda is responsible for determining whether the browser (Only google chrome accepts webp images) is accepting WEBP images from request headers
2. One-time Image Resizer — When an image is uploaded to s3 we convert this image to a standard 1500*1500 image and move the originally uploaded image to amazon glacier
3. Run-Time Image Resizer — When the request misses CloudFront cache, this lambda converts the standard image present on s3 to requested image and stores it in the CloudFront cache.

Glacier is almost 5 times cheaper than S3 but it takes hours to fetch data from a glacier. We send the original high-resolution image to Glacier and keep a copy of a standard 1500*1500 image converted by One-time Image Resizer in s3. This image is perfect for our runtime image version creation.

Request flow
We store all the configurations(height, width, watermark, crop, white space shrinking etc.) of an image in the database and keep one copy of it in s3 document, after every creation or updating of configuration we update the s3 file.
The request first comes to webp checker it appends .webp extension on the basis of request headers. If CloudFront has an image with this URL it returns the image otherwise request goes to Run-Time Image Resizer it fetches the standard image from s3 and configuration from s3 document, by looking at the extension of URL (webp images will have URL like .jpg.webp) it converts the image and stores it in the CloudFront cache

Useful links:

Engineering @ Housing/Proptiger/Makaan

Engineering and technology articles from developers

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store