Does Serverless work at Scale?

Vaibhav Puranik
Feb 25 · 3 min read

With the advent of cloud and Serverless technologies the number of architectural options available in building systems have increased by many fold. I see many teams struggling to make decisions on whether to use serverless or containers based systems in the organization. My objective is to present you one real life case we encountered at GumGum and the thought process behind choosing containers over serverless.

Verity - Headless Content Extractor

GumGum takes pride in it’s contextual intelligence capabilities. Our proprietary contextual intelligence API Verity uses Natural Language Processing in combination with Computer Vision to derive context and brand safety information for a webpage. Here is the simplified architecture of the system:

Image for post
Image for post
GumGum’s Proprietary Contextual Intelligence System — Verity

When Verity API gets a page url, it sends the url to HCE or Headless Content Extractor. The objective of headless content extractor is to download the page and extract HTML of the page. HCE also finds out all the images in the HTML. HTML content of the page is then sent to TAPAS (Gumgum’s Natural Language Processing system) to determine the context and brand safety information. Extracted images are sent to VERTEX (GumGum’s Computer Vision system) to determine the context and brand safety information. The information returned by both the systems is then merged into Verity API to derive the inference about context and brand safety. This technique of merging two different inferences into one is also known as Late Fusion.

HCE is a very critical component in the entire system. Accurate inference depends upon extracting correct HTML and text on a page. When we began working on HCE, our first choice was to use AWS Lambda for the purpose. It was pretty obvious, just send each URL to a lambda! You will not have to worry about scaling and maintaining the infrastructure. Pretty quickly our engineers developed the lambda that will use a headless browser to download the page and extract the content. When we started testing the system, we realized that on an average a HCE execution for a page takes 30 seconds! The headless browser has to start, load the page, wait for the DOM to load and then extract the HTML. It turns out that most of the web pages on internet load really slowly and they contain a lot of third party javascript libraries which takes long time to execute.

Cost of Scale

The team then decided to evaluate the cost of their approach. They decided to calculate the cost of lambda at the intended scale. We were expecting to process 180 MM pages per month. Each page would approximately take 30 seconds as described above. We need 1 GB of memory in Lambda to accomplish the HTML extraction. Based on this, the team calculated the following numbers:

Image for post
Image for post
Cost of Serverless function execution for HCE for a month — serverlesscalc.com

As you can see, the cost is approximately $90,000 per month! In this money we can buy 350 c5.2xls for 24 x 7 for a month without any reservation! If we decide to use spot instances (which we did), we can buy 650 c5.2xls for a month! c5.2xls have 8 VCPUs and 16 GB of memory. We could have run 8 headless browsers easily on one machine. Doing some calculation, we realized that we only need to run 260 c5.2xls to serve our scale. The cost of running 260 instances with EBS drives is approximately $40,000!

If you go little deeper and understand how AWS Lambda pricing works, you will realize that it’s not the number of invocations that’s increasing the cost. It’s 30 second processing time that’s increasing the cost. Number of requests for lambda are very cheap, but compute is not cheap. It’s the compute time that makes lambda an expensive option for Verity HCE.

Conclusion

I wouldn’t rule out AWS Lambda for many other applications where processing time is very short. In such cases, it’s better to use Lambda instead of managing container based environment yourself. Managing container based environment has its own set of headaches (such as managing Kubernetes)!

gumgum-tech

Thoughts from the GumGum tech team

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store