Does Serverless work at Scale?
With the advent of cloud and Serverless technologies the number of architectural options available in building systems have increased by many fold. I see many teams struggling to make decisions on whether to use serverless or containers based systems in the organization. My objective is to present you one real life case we encountered at GumGum and the thought process behind choosing containers over serverless.
Verity - Headless Content Extractor
GumGum takes pride in its contextual intelligence capabilities. Our proprietary contextual intelligence API Verity uses Natural Language Processing in combination with Computer Vision to derive context and brand safety information for a webpage. Here is the simplified architecture of the system:
When Verity API gets a page url, it sends the url to HCE or Headless Content Extractor. The objective of headless content extractor is to download the page and extract the HTML of the page. HCE also finds out all the images in the HTML. HTML content of the page is then sent to TAPAS (GumGum’s Natural Language Processing system) to determine the context and brand safety information. Extracted images are sent to VERTEX (GumGum’s Computer Vision system) to determine the context and brand safety information. The information returned by both the systems is then merged into Verity API to derive the inference about context and brand safety. This technique of merging two different inferences into one is also known as Late Fusion.
HCE is a very critical component in the entire system. Accurate inference depends upon extracting correct HTML and text on a page. When we began working on HCE, our first choice was to use AWS Lambda for the purpose. It was pretty obvious, just send each URL to a lambda! You will not have to worry about scaling and maintaining the infrastructure. Pretty quickly our engineers developed the lambda that will use a headless browser to download the page and extract the content. When we started testing the system, we realized that on an average a HCE execution for a page takes 30 seconds! The headless browser has to start, load the page, wait for the DOM to load and then extract the HTML. It turns out that most of the web pages on internet load really slowly and they contain a lot of third party javascript libraries which takes long time to execute.
Cost of Scale
The team then decided to evaluate the cost of their approach. They decided to calculate the cost of lambda at the intended scale. We were expecting to process 180 MM pages per month. Each page would approximately take 30 seconds as described above. We need 1 GB of memory in Lambda to accomplish the HTML extraction. Based on this, the team calculated the following numbers:
As you can see, the cost is approximately $90,000 per month! In this money we can buy 350 c5.2xls for 24 x 7 for a month without any reservation! If we decide to use spot instances (which we did), we can buy 650 c5.2xls for a month! c5.2xls have 8 VCPUs and 16 GB of memory. We could have run 8 headless browsers easily on one machine. Doing some calculation, we realized that we only need to run 260 c5.2xls to serve our scale. The cost of running 260 instances with EBS drives is approximately $40,000!
If you go little deeper and understand how AWS Lambda pricing works, you will realize that it’s not the number of invocations that’s increasing the cost. It’s 30 second processing time that’s increasing the cost. Number of requests for lambda are very cheap, but compute is not cheap. It’s the compute time that makes lambda an expensive option for Verity HCE.
Conclusion
I wouldn’t rule out AWS Lambda for many other applications where processing time is very short. In such cases, it’s better to use Lambda instead of managing container based environment yourself. Managing container based environment has its own set of headaches (such as managing Kubernetes)!
