Building High Performance Applications — Lessons Learnt, Part — 2

Welcome to the Part 2 in the series of my posts on “Building High Performance Applications”. For those who are on the path to building a high performance and a massively scalable application, this post will give a recap of the evolution of software architectures and in the next post, Part 3, I will be decomposing an application architecture for creating highly concurrent, high throughput services that will fit right into your SOA or Microservices architecture.

In Part 1 of this series, I started with identifying some of the most important performance bottlenecks that you face while building such an application server. But don’t worry if you have missed that. I will let you know when it will be the right time to read it again. Part 1 can be found here: Building High Performance Applications, Lessons Learnt — Part 1.

Before getting to the agenda of Part 2, I would like to share some interesting data with you so that it will help you understand what we are calling “High Performance” and how many concurrent network requests does “Massively Scalable” mean.

Netflix — Scale of API Calls

The data I found out was dated, January 2012 and when Netflix was handling 42 Billion API calls every month. Interestingly,

42 Billion API Calls / Month ~= 1.4 Billion API Calls / Day ~= 58 Million API Calls / Hour ~ = < 1 Million API Calls / Minute ~= 16 K API Calls / Second

So it would be safe to assume that, Netflix was handling ~ 16 K Concurrent API calls back in 2012. I can only imagine what would be the scale now 6 years later for a company with a 70x growth curve.

Amazon — Scale of Search API

According to another data I found, which was dated 2016 when in an interview Amazon CEO Jeff Bezos said that the E-Commerce giant was handling ~ 1.1 Million API Calls / Second.

Pretty Cool!! Here’s some more.

• Instagram reportedly had 500 million daily active users in 2017.
• Paytm has a monthly avg traffic of ~100 million on desktop and mobile.
• I want to mention Uber here, but I think I’ve already made my point.

There’s a lot more data available on the internet and I can continue giving examples of how important the scale of applications is today. But I am sure you have a pretty good idea of how the scale of applications has become so important in the last few years especially with the emergence of cloud and SaaS as a preferred delivery model. Sadly, the traditional approach towards building web applications no longer works or has very limited area where it can be utilized.

A Web application usually comprises of 3 layers:

1. Presentation Layer
2. Business Logic Layer
3. Persistence / Database Layer

Until a decade back, 2-tier architectures of client-server applications were more prevalent. In a 2-tier architecture, the Business logic layer and Persistence layer are tightly coupled and usually sit behind a web server. Web servers like Apache being process based models, would fork a different process for each client request, thus being extremely resource hungry. And also due to the tight coupling between business logic and database, changing something in the system proved a costly affair.

3-tier/N-tier architecture was adopted in which Presentation Layer, Business Logic Layer and the Persistence layer were decoupled and put on separate physical tiers. This allowed change in the application a lot easier to implement. Building applications became faster since separate teams could now be utilized for building each tier independently. The decoupling allowed each tier to be scaled separately, however, there still were a few shortcomings.

Shortcomings in 3-tier Architecture

• It did not consider the nature of the applications. Whether it was an E-Commerce application, a payment application or a video streaming application, the application logic was still coupled in a single layer.
• It was designed in the era when Cloud and elastic scalability was still in the making. It was not possible to scale specific parts of the application layer.
• It was designed in the era when mobile phones and handheld devices were not so common and web applications were still accessed on desktop browsers.
• Software was getting outdated with emerging advancements in multi core CPUs available on demand. Process based web servers were too old to cope up with modern CPU architectures. Newer thread based models were already being rolled out that offered much higher throughput. Performance demands were also increasing.

Emergence of Service Oriented Architectures

A service oriented architecture is built on top of a 3-tier architecture and sees an application as a set of services and applications instead of decoupled tiers. It addresses the shortcomings of a 3-tier architecture.

A service implements some functionality which is used by applications and possibly other services. Visualize it this way : In SOA, an Application may be thought of as the presentation layer, while an SOA Service may be thought of as a business logic layer or a database layer.

Services communicate with each other using standard network protocols like HTTP / REST. The message formats are well defined using metadata of sorts.

Now What Is a Microservices Architecture then?

Somethings are just a buzzword. Let me make it easier for you.

The only difference between SOA and Microservices is the size and scope of the services. A Microservices architecture is only a kind of SOA that defines services to be way too finely grained. It evolved only due to DevOps practices of Continuous Integration and Continuous Delivery. It requires services to be independent application components, where as SOA services can belong to same application component.

The real question however is :

1. How to decompose an application into services?
2. How to implement a service, for high performance and make your system massively scalable?

Wrapping Up Before Getting To Part 3

Alright!! I will end this post right here and give time to the readers to ponder over the 2 questions that have remained unanswered in this post. And also to prepare yourself to take a deep dive into creating a multi threaded architecture that takes a staged approach to event driven concurrency.

Do like / share / comment, or just put your thoughts across to me. I’d love to improve my writing skills. You can reach me @ https://lightwsd.com