With the advent of cloud and Serverless technologies the number of architectural options available in building systems have increased by many fold. I see many teams struggling to make decisions on whether to use serverless or containers based systems in the organization. My objective is to present you one real life case we encountered at GumGum and the thought process behind choosing containers over serverless.
Verity - Headless Content Extractor
GumGum takes pride in it’s contextual intelligence capabilities. Our proprietary contextual intelligence API Verity uses Natural Language Processing in combination with Computer Vision to derive context and brand safety information for a webpage. Here is the simplified architecture of the system:
When Verity API gets a page url, it sends the url to HCE or Headless Content Extractor. The objective of headless content extractor is to download the page and extract HTML of the page. HCE also finds out all the images in the HTML. HTML content of the page is then sent to TAPAS (Gumgum’s Natural Language Processing system) to determine the context and brand safety information. Extracted images are sent to VERTEX (GumGum’s Computer Vision system) to determine the context and brand safety information. The information returned by both the systems is then merged into Verity API to derive the inference about context and brand safety. This technique of merging two different inferences into one is also known as Late Fusion.
Cost of Scale
The team then decided to evaluate the cost of their approach. They decided to calculate the cost of lambda at the intended scale. We were expecting to process 180 MM pages per month. Each page would approximately take 30 seconds as described above. We need 1 GB of memory in Lambda to accomplish the HTML extraction. Based on this, the team calculated the following numbers:
As you can see, the cost is approximately $90,000 per month! In this money we can buy 350 c5.2xls for 24 x 7 for a month without any reservation! If we decide to use spot instances (which we did), we can buy 650 c5.2xls for a month! c5.2xls have 8 VCPUs and 16 GB of memory. We could have run 8 headless browsers easily on one machine. Doing some calculation, we realized that we only need to run 260 c5.2xls to serve our scale. The cost of running 260 instances with EBS drives is approximately $40,000!
If you go little deeper and understand how AWS Lambda pricing works, you will realize that it’s not the number of invocations that’s increasing the cost. It’s 30 second processing time that’s increasing the cost. Number of requests for lambda are very cheap, but compute is not cheap. It’s the compute time that makes lambda an expensive option for Verity HCE.
I wouldn’t rule out AWS Lambda for many other applications where processing time is very short. In such cases, it’s better to use Lambda instead of managing container based environment yourself. Managing container based environment has its own set of headaches (such as managing Kubernetes)!