Eat My Dust! : Making of Ballerina’s Lightning Fast HTTP Transport
After years in the making, it is about high time I talked a bit about the code base or the component that could literally control the life of an integration engineer. If you don’t write it right, it could write your fate. It will be tested in different territories but yet expected to perform without a glitch. It is none-other than HTTP Transport, which is a must in any network oriented program. This article talks a bit about five key aspects which went into the making of lightning fast and feature rich HTTP Transport of Ballerina.
Firstly, How Fast It Is ?
Below picture gives you a relative idea!
These tests were done using a network of three identical machines. One machine was used for the client, one was used for Gateway (Envoy/HTTP Transport)and the other was used for the echo-backend. Machine details are as follows model Name - ThinkPad-T540, processor name - Intel Core i7, processor speed - 2.6 GHz, total number of cores - 8, memory - 8GB. Each Gateway was hit with 100 concurrent users with 1K messages.
Now that you have some sense on how fast it is. Let’s have a look into five key aspects which made it possible.
1) Hard-Knock Experience
Debugging an issue to figure out why the message was corrupted while your colleagues playing cricket during the annual company trip is like the worst thing that could happen to a South Asian and yeah! it happened to us. Walking in late to one of your best friends wedding because you wanted to figure out why the hell transport threads hung is a sure way to lose your Friend. Trust me. Walking out in the middle of a movie because transport threads started spinning, well, again not the best thing that could happen in your life. And the list goes on.
These are all because of issues, popped out from a high performant HTTP transport written years ago. But on the bright side, these issues molded the will to get the new HTTP Transport right while giving the required background knowledge on how to test the new HTTP transport, which is not written in books.
2) Make It Work, Then Make It Fast
This is a principle we followed through out the development of the new transport. This transport has 30 odd features starting from supporting HTTP, HTTP2 and WebSockets to throttling-requests capability to make sure Ballerina never goes out of memory. At the beginning the focus was on making sure these features works as expected under any condition and then to make it fast. Doing otherwise, could end up in premature optimization which usually results in overly smart and complex code. This type of code is less readable and error prone. A nightmare to engineers who maintain the code.
Not to mention, while working on all these features, we also had to make sure that we follow the below principle. So completing features itself was so no easy task.
Be conservative in what you do, be liberal in what you accept from others
This was needed because HTTP is roughly 30 years old and we have all sorts of implementations of this protocol which are slightly or majorly off from the spec!
3) Simplicity is The Ultimate Sophistication
Leonardo Da Vinci said this and we strongly believe in this. Well, how simple it is, is respective to the problem domain you work with. However, The more you know about the problem domain, The better you can simplify the solution. Luckily, we had a team which knew everything it needs to know. As a result I believe we could come up with a simple, readable code base while maintaining performance. We tried our best to not to break OOP principles such as Open-Closed Principle, Dependency Inversion Principle, Single Responsibility Principle, etc. Moreover, this code base is heavily influenced by Clean Code Book and State Design Pattern.
we must mention the role Netty Framework played in this regard. Without its elegant Pipeline Pattern and Reactive Style, we couldn’t have come this far!
4) Reactive Style of Coding
This style of programming evolves around the propagation of change. Whenever there is a state change, program propagates the change to all the interested parties so that they can react to the change. Netty provided a nice way to do this with Futures while hiding all the complexities of Java NIO. Inspired by this design, we modeled all our APIs using Futures. This allowed the entire transport to be completely none-blocking while maintaining the simplicity. In other words, if you want you can write a simple reverse proxy with this transport, which only requires one thread to execute.
This allowed us to completely get rid of all the synchronization problems. Which means we have,
- No lock contentions
- No context switching problems
- Less Thread scheduling required
- Less CPU cache misses
- No Amdahl’s Law effect
All of these are key contributors for the performance of lightning fast HTTP transport.
If you want to know more about Amdahl’s Law click here!
5) Collaborative Team
Last but not the least, we have the best collaborative team,
- Vijitha/Bhashinee — Can’t hear a thing that comes out of their mouth but they make sure “actions speak louder than words”
- Rukshani — Would choose reading specifications over eating cake
- ChamilE — Waste a lot of time on sports but at the same get the job done (I guess I do the same :P)
- Riyafa — Makes sure everything is in order, even the commit messages
- Chanaka — Don’t mind copying things all day from machine to machine till the issue is resolved
Also, Isuru Udana, Isuru Ranavaka, Irunika and few other engineers contributed to this effort in the past.
We need to appreciate everyone’s effort on this. Because without each member’s contribution, we couldn’t have make it this far. Also, there is a long way to go. Currently we are working on the last bits of HTTP2 features and we will continue to improve the code base which will be a continuous process!
📝 Read this story later in Journal.
🗞 Wake up every Sunday morning to the week’s most noteworthy Tech stories, opinions, and news waiting in your inbox: Get the noteworthy newsletter >
