Benchmark blockchain using Hyperledger Caliper
Benchmark a private blockchain with ERC20 transfer
Knowing the capabilities of our ecosystem is important as well as creating and maintaining a private blockchain. These results will help us to understand whether the system meets our requirements or will provide a huddle free service to the users. If not we can always scale it.
In this article we are going to benchmark a private blockchain with IBFT2.0 consensus mechanism created using Hyperledger besu. If you are not familiar with it please find an article here:
To benchmark our private blockchain we are using another tool under Hyperledger umbrella named Caliper.
Hyperledger Caliper is a blockchain benchmark tool, it allows users to measure the performance of a blockchain implementation with a set of predefined use cases. Hyperledger Caliper will produce reports containing a number of performance indicators to serve as a reference when using the following blockchain solutions: Hyperledger Besu, Hyperledger Burrow, Ethereum, Hyperledger Fabric, FISCO BCOS, Hyperledger Iroha and Hyperledger Sawtooth.
So currently a private blockchain with 4 nodes are running. Now we have to configure some connectors and benchmarking scenarios. Hyperledger has provided some examples for different cases in this repo. But this is more of a general example. So just for fun I am going to make some changes in this so that we can test ERC20 token transfer characteristics of our private blockchain.
So I took a clone of this repo and made some changes. Final code can be found here. Before getting into more details if you are interested you can check the documentation of Hyperledger Caliper. Now we can dive into the process. We can use either source code of Caliper or docker image. I am going to use docker image of Caliper for benchmarking. So I have included a docker-compose file in the root that looks like this.
There are some env variables applied in this that you may have to consider. CALIPER_BIND_SUT is to specify the System Under Test (SUT). Currently caliper supports only a bunch of networks. We have specified it as Besu because we are benchmarking a network created by Hyperledger Besu. Then there are two important file mentioned here, CALIPER_BENCHCONFIG and CALIPER_NETWORKCONFIG. Benchconfig points to the file which contains the test cases or benchmarking scenarios. Network config specifies the SUT. We will look into both these later.
Another variable is CALIPER-FLOW-SKIP-INSTALL. Generally there is an order for the benchmarking process of Caliper which is specified as init, install, start, test and end. Under the hood Caliper is deploying a predefined smart contract to the network and executing some function calls to this contract. By monitoring these contract function calls caliper generates a report. In the example given by Hyperledger they are provided a simple contract. It only have 3 functions create, modify and query. When we run caliper using that example it will first deploy this contract to SUT and execute these functions in different conditions. That is the general way of testing. But what about if want to use our own smart contracts? That is also possible. We can add our contract details into configurations. But Caliper has a small drawback in this area. We cannot use smart contracts that accepts arguments in constructor, because there isn’t any provision to pass arguments while deploying the smart contract given that deployment is managed by Caliper. A solution to this problem is to deploy a contract by ourselves and give the details to caliper. That what we are doing here. In that situation we have to inform caliper that you don’t have to deploy any contracts and that is done by CALIPER-FLOW-SKIP-INSTALL.
That’s a lot to digest! Anyway we can continue. Moving into the core there are three things what we have to consider in the code.
Benchmark configuration file
The benchmark configuration file describes how the benchmark should be executed. It tells Caliper how many rounds it should execute, at what rate the TXs should be submitted, and which module will generate the TX content. It also includes settings about monitoring the SUT.
In this case we have specified two test cases, 100 transactions with a linear rate of 10 –100 transactions per second and 100 transactions at a fixed rate. You can configure this to your preferences. There more types other that linear-rate and fixed-rate. You can try that also. Consider to start with smaller values. Other wise there is a chance that your node will be overloaded. Don’t panic! Just stop the caliper and restart the node. You are ready to test again. Anyway we are going with this configuration for the time being.
Network configuration file
The content of the network configuration file is SUT-specific. The file usually describes the topology of the SUT, where its nodes are (their endpoint addresses), what identities/clients are present in the network, and what smart contracts Caliper should deploy or interact with.
First we have specified the connector as “ethereum” since besu is a fork of ethereum. Then SUT specifics are given. Please remember to expose the socket rpc url in your connecting node. Because socket connection is more reliable than http rpc connection when coming to benchmarking.
We need to deploy an ERC20 contract to the network before editing this file. Then provide the addresses here in the config. Also need to specify the gas limit for the function and abi of the contract. (I left it empty in this image because its lengthy!!!)
Workload modules
Workload modules are the brain of a benchmark. Since Caliper is a general benchmark framework, it does not include any concrete benchmark implementation. When Caliper schedules TXs for a given round, it is the task of the round’s workload module to generate the content of the TXs and submit it. Each round can have a different associated workload module, so separating your workload implementation based on phases/behaviour should be easy.Workload modules are simply Node.JS modules that must export a given factory function. Other than that, the workload module logic can be arbitrary. Really, anything you can code in Node.JS.
Since it is the brain I haven’t changed much on that for now. Even though small changes are done in it, that you can find it yourself.
This is the implementation of the transfer function that we are using for testing. At line 30 you can see how the function is interacting with contract using the connector.
This is a helper file for transfer.js. We have specified a single address as target because we are sending all 100 txns to the same account. If you want to you can put some logic so that it will send 100 txns to 100 different accounts. The hyperledger example have such a logic in it.
This is another helper and it is more connector specific. At line 98 we have a function to generate the smart contract function call request.
Now we all set. Execute docker-compose up to run the caliper. Check the logs of caliper and the node too. Just to make sure we are not overloading our node. After successful execution you will get something like this in the console.
And, a report like this.
Thanks for reading….!!
Github repo: https://github.com/prasanth0105/caliper-benchmarks-erc20
Reference: https://hyperledger.github.io/caliper/
