What is your TPS ?

There are hardly any conversations and meetings on Blockchain that go without bringing up the topic of performance and scalability. In computer science, the most common metrics that comes to anyone’s mind related to performance is: throughput or transactions per second (aka TPS) of the system. A generally accepted conclusion is that; the higher the TPS, the more performant the system is.

Unfortunately, that by itself, is a misleading indicator of the performance and it is even more misinterpreted in Blockchain based systems.

In order to draw a common life analogy, imagine three kids in a jump rope competition as in the image above. In this competition, if the the only winning criteria was the number of jumps per minute; the obvious conclusion would be that the more number of jumps in one minute, the more performant the kid is.

However, as you can see in the image, each kid is jumping at a different height? Would you say that the number of jumps per minute is still the only winning criteria if the height of the jump was also a consideration? Would your answer be different if each kid had a different rope length? Or if the surface of the jump was different for each kid? Most likely, you will refine your answer to include those considerations, in addition to “jumps per minute”.

This basic example illustrates two main considerations in accurately and uniformly measuring performance of a system: (1) standard definition of the metrics (2) standard process to measure/monitor those metrics.

As it turns out, in Blockchain space, there is neither a single standard, accepted definition of a transaction nor a standard process to measure its granularity. Each Blockchain provider defines the transaction, its boundaries, granularity and process to measure that in its own (selfish) way.

In contrast, traditional databases/Online transaction Processing Systems the transaction benchmarking criteria have been defined and approved by a standards body, Transaction Processing Performance Council ( TPC). (Excerpt from TPC’s website). “TPC focuses on two major activities: (1)creating good benchmarks and (2) creating a good process for reviewing and monitoring those benchmarks. Good benchmarks are like good laws. They lay the foundation for civilized(fair) competition.” Unfortunately, these benchmarks( or laws) are still not well defined for Blockchain implementations.

Further, even with the existence of well defined standards and criteria, performance benchmarking is an advanced engineering effort. It requires a very meticulous calibration of multiple components to ensure an optimal load on the system being benchmarked. Even a single mis-configured component during benchmarking exercise can lead to skewed results and interpretations.

In summary, given the lack of standards, independently reviewed and accepted benchmark for a transaction definition and a widely accepted measurement process in Blockchain space, perhaps the best answer when asked how many TPS your platform can support is — it depends -:)