Waves Enterprise Completes Performance Tests
The tests have proven the platform’s stable performance under different scenarios, confirming target throughput parameters.
The Waves Enterprise platform has been put through a series of performance tests.
The main goal of the tests was to confirm the platform’s high throughput under real-world operational conditions and its stable performance under given loads, as well as to formulate recommendations for network configurations based on specific platform operation scenarios.
Tested target parameters included:
- Throughput of 1,000 transactions per second (tps) for transfer transactions
- 16 Mbps of data
- 100 smart contract invocations per second
The tests aimed to:
- Emulate the topology and configuration of a blockchain network that is as close as possible to those of industrial solutions
- Emulate three different scenarios of blockchain operation with respect to load profiles
- Run tests for specific load profiles and time periods
- Collect data on network throughput, as well as other metrics characterizing software operation in different scenarios
- Determine factors that restrict network performance in case the tests reveal insufficient performance.
The load testing contour was configured in a way that was as close to our customers’ production contours as possible.
Blockchain nodes were deployed in Docker containers managed by the orchestrator Kubernetes. Since the orchestrator dynamically redistributes system resources based on container load, we were able to achieve the operational conditions under which apps operate in a dynamically changing environment.
Resource limit on a node container was 4 CPUs and 16 GB RAM, while the load was directly addressed to the node’s TCP ports.
Network configurations for various scenarios
For platform tests, the Proof of Authority (PoA) consensus algorithm was used.
- For transfer transactions, the round duration was 8 seconds and the block size 1.75 MB
- For data transactions, the round duration was 1 second and the block size 4 MB
- For smart contracts the round duration was 10 seconds and the block size 4 MB
For load testing, we emulated three scenarios for platform operation: a payment solution, data transfer and storage, and business process automation. These scenarios are used in our existing projects.
To emulate a payment solution’s operation, we used a token transfer transaction profile for network load. In a payment solution, each connected participant, such as a bank or a payment system, has equal rights. We therefore sent transactions in packages of 2,500 to each network node to avoid having an empty pool of unconfirmed transactions on each node. Each transaction’s size was roughly 600 bytes.
A blockchain solution enabling data storage and transfer can be used to create distributed ledgers for various documents, as well as, for instance, for collection and subsequent use of data from IoT devices. For this scenario, the quantity of transferred data is vital, rather than the number of transactions per second. To emulate this scenario, we used data recording transactions sent in packages of 100, with a total transaction size of around 50 KB.
The third scenario emulated the operation of a business process automated on the blockchain, such as a product supply chain or automation of electricity payments. To emulate this scenario, network load was supplied by smart contract invocation transactions of various sizes: 0.1 KB, 10 KB, 50 KB and 100 KB. This was done to emulate the operation of smart contracts of various complexity and different payloads transferred as an invocation argument, as well as to determine the correlation between smart contract complexity and network throughput.
In addition, to compare the interaction speed between the smart contract’s container and node under different conditions, both smart contracts that call the node’s state and those which do not were used.
For each scenario, the load testing period was 8 hours. As no incidents were reported during that period and no decline in the throughput of the nodes and the overall blockchain was detected, we believed this period was sufficient.
Outcome and conclusions
The diagram below summarizes the results of our load tests. We can confidently say that the platform meets throughput requirements for various scenarios requested by our customers.
The graphs display the correlation between the time of smart contract use and network throughput from its complexity and payload transferred to it.
Based on the testing results, the following conclusions can be drawn for each scenario.
The current throughput of 1,000 tps is “cruise”-mode, meaning that in this mode, resources allocated to the node are more than sufficient and it operates stably. To further increase throughput, platform users can lift a limit of 500 transactions per microblock. This limit was established as recommended for a mixed platform load scenario to avoid the creation of overly large blocks, which in some cases could trigger unstable network interaction and node behavior.
The test confirmed that a throughput of 16 Mbps is a standard regime for platform operation.
In this scenario, blockchain operation imposes extra requirements on computational resources, primarily the disk subsystem. For stable node operation under this scenario, fast HDD or SSD disks are preferable.
To further increase network throughput, horizontal node scaling can be applied. Also, the Waves Enterprise team is constantly working on optimizing resource-consuming operations.
Smart contracts are involved in all business process automation cases on the blockchain. It was therefore vital to evaluate the platform’s throughput while processing smart contracts.
We tested various types of smart contract, with all of them showing changes in throughput, depending on input data.
Based on analysis of resource consumption in the test contours, as well as in the public network’s nodes where smart contracts operate under load, we recommend scaling nodes for memory. Additionally, one problematic area is the operation time of a smart contract itself, as the throughput potential facilitates increasing the number of processed transactions by several dozen times over.
During load testing under this scenario, the network throughput’s dependence on smart contract execution time and payload supplied as arguments was detected.
Predictably, the maximum throughput was achieved for “fast” smart contracts with a small payload of 0.1 КB. Based on our experience, for most business process automation cases on the blockchain, smart contract invocation transactions of 2 KB to 4 KB are used.
Given this, and the fact that network throughput dependence on payload size is non-linear, it’s safe to say that for most cases, throughput will be close to maximum.
For more stable smart contract operation under load, attention should be paid to the allocation of sufficient RAM to the machine on which smart contracts are executed.
Plans for further development
From a platform development viewpoint, smart contract development is one of Waves Enterprise’s priorities, and their speed increases with each release. Currently, several solutions are at the R&D or prototyping stage, and will likely lead to a several-fold increase in network throughput for this scenario.
You will be able to learn more on testing methodology and outcomes in a detailed load testing report that we will publish shortly.