Modifying the Batch Size in Hyperledger Fabric v2.2

Modifying the Batch Size in Hyperledger Fabric

1. Overview

Batch Size and Batch Time are one of the important factors to achieve high throughput in Hyperledger Fabric. These factors decide the Transaction Per Second (TPS) of the blockchain network. Also one of the important aspects is to design the network efficiently. It has been observed that the use of a LevelDB world state database is seen to enable higher throughput and lower latencies with small batch sizes, though this benefit is lost with large batch sizes. There are other factors also which contribute to TPS. But this article is more toward how we can update the batch size of the running Hyperleder fabric network. I am using test-network v2.2 for this implementation.

Check out my course on Hyperledger Fabric Deployment on MultiHost

Note: The steps are exactly the same for updating the batch timeout.

2. Terminologies

• Batch size - It defines the size of transactions in a block. No block will appear larger than absolute_max_bytes large or with more than max_message_count transactions inside the block. If it is possible to construct a block under preferred_max_bytes, then a block will be cut prematurely, and transactions larger than this size will appear in their own block.
• Batch timeout. The amount of time to wait after the first transaction arrives for additional transactions before cutting a block. Decreasing this value will improve latency, but decreasing it too much may decrease throughput by not allowing the block to fill to its maximum capacity.
• Max Message Count: The maximum number of messages to permit in a batch

3. Understanding the Channel Update Process

1. fetch the config block from mychannel
2. convert it into JSON and extract the config portion.
3. locate the configuration you need to modify, here in my case, batch size 10 => say 20.
4. convert two files into 2 PB files, and compute the delta.
5. decode the PB file again, add envelope, and encode it again.
6. Use ordererOrg Admin to sign the result of (5).

This is different from add org3 tutorial as ordererAdmin is needed to sign this update.

4. Network: Setup

For this demonstration, I am using the test-network provided in the examples. I have added few shell scripts and removed the unused code, use can use this repository.

we will start the network with mychannel, I am using docker images with tag 2.2

cd fabric-samples-advance-topics/test-network
./network.sh up createChannel -c mychannel -i 2.2

Network: Setup-Modifying the Batch Size in Hyperledger Fabric

Once the network is up and running we will start the cli container to interact with the network.

docker-compose -f docker/docker-compose-cli.yaml up -d

Read Adding a new Orderer in Running Hyperledger Fabric v2.2 Network

5. Channel Update

This is a very important step in this we will be updating the mychannel and change the batchsize from 10 to 20. Initially, the batch size is 10 but we will update it to 20, we can see the batchsize at configtx/configtx.yaml.
Steps in the section are what we discussed in section 3.

1. Fetch the latest configuration
   we’ll pull the channel configuration in protobuf format, creating a file called config_block.pb.

docker exec -e CHANNEL_NAME=mychannel cli sh -c ‘peer channel fetch config config_block.pb -o orderer.example.com:7050 -c $CHANNEL_NAME — tls — cafile $ORDERER_CA’

Fetch the latest configuration-Modifying the Batch Size in Hyperledger Fabric

2. Convert the Configuration to JSON
We will convert the latest block config_block.pb , which we go from the previous step into JSON and will trim the required part into the config.json file

docker exec cli sh -c ‘configtxlator proto_decode — input config_block.pb — type common.Block | jq .data.data[0].payload.data.config > config.json’

we can check that using
docker exec cli sh -c ‘ls -lh’

Convert the Configuration to JSON-Modifying the Batch Size in Hyperledger Fabric

3. Check the current value
we will find the current value of blocksize and will compare it with the updated value. We have to export the MAXBATCHSIZEPATH so that we can read it and change the value

MAXBATCHSIZEPATH=”.channel_group.groups.Orderer.values.BatchSize.value.max_message_count”
docker exec -e MAXBATCHSIZEPATH=$MAXBATCHSIZEPATH cli sh -c ‘jq “$MAXBATCHSIZEPATH” config.json’

Check the current value-Modifying the Batch Size in Hyperledger Fabric

4. Updating the value
Now we will be updating the value from 10 to 20 and will store that in the temporary file modified_config.json .

docker exec -e MAXBATCHSIZEPATH=$MAXBATCHSIZEPATH cli sh -c ‘jq “$MAXBATCHSIZEPATH = 20” config.json > modified_config.json’

Updating the value-Modifying the Batch Size in Hyperledger Fabric

we can check the content of modified_config.json with the updated value.
docker exec -e MAXBATCHSIZEPATH=$MAXBATCHSIZEPATH cli sh -c ‘jq “$MAXBATCHSIZEPATH” modified_config.json’

5. Converting JSON to ProtoBuf

In this step, we are converting the initial config.json and modifed_config.json (with updated value)into ProtoBuf (PB)format.

docker exec cli sh -c 'configtxlator proto_encode --input config.json --type common.Config --output config.pb'
docker exec cli sh -c 'configtxlator proto_encode --input modified_config.json --type common.Config --output modified_config.pb'

Converting JSON to ProtoBuf-Modifying the Batch Size in Hyperledger Fabric

6. Delta Calculation
Now, we have to calculate the delta between the two .pb files which we just generated.

docker exec -e CHANNEL_NAME=mychannel cli sh -c 'configtxlator compute_update --channel_id $CHANNEL_NAME --original config.pb --updated modified_config.pb --output final_update.pb'

Delta Calculation-Modifying the Batch Size in Hyperledger Fabric

7. Adding the update to the envelope
Now we have to calculated the difference between the old config and the new one, we can apply the changes to the config

docker exec  cli sh -c 'configtxlator proto_decode --input final_update.pb --type common.ConfigUpdate | jq . > final_update.json'
docker exec  cli sh -c 'echo "{\"payload\":{\"header\":{\"channel_header\":{\"channel_id\":\"mychannel\", \"type\":2}},\"data\":{\"config_update\":"$(cat final_update.json)"}}}" | jq . >  header_in_envolope.json'
docker exec cli sh -c 'configtxlator proto_encode --input header_in_envolope.json --type common.Envelope --output final_update_in_envelope.pb'

Adding the update to the envelope-Modifying the Batch Size in Hyperledger Fabric

8. Signing the update
Now we have to sign this update proto as Orderer. Remember that we exported the necessary environment variables to operate

docker exec cli sh -c 'peer channel signconfigtx -f final_update_in_envelope.pb'

Signing the update-Modifying the Batch Size in Hyperledger Fabric

9. Initiate the Update command
Now the update process is done through orderer and for that, we have to set up a few environment variables that will be pointing to the OrdererAdmin.

CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/tls/ca.crt
CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/users/Admin@example.com/msp
CORE_PEER_ADDRESS=orderer.example.com:7050

updating the channel, Our config update transaction represents the difference between the original config and the modified one, but the ordering service will translate this into a full channel config.

docker exec -e CORE_PEER_LOCALMSPID=$CORE_PEER_LOCALMSPID -e CORE_PEER_TLS_ROOTCERT_FILE=$CORE_PEER_TLS_ROOTCERT_FILE -e CORE_PEER_MSPCONFIGPATH=$CORE_PEER_MSPCONFIGPATH -e CORE_PEER_ADDRESS=$CORE_PEER_ADDRESS -e CHANNEL_NAME=mychannel cli sh -c ‘peer channel update -f final_update_in_envelope.pb -c $CHANNEL_NAME -o orderer.example.com:7050 — tls — cafile $ORDERER_CA’

Initiate the Update command-Modifying the Batch Size in Hyperledger Fabric

10. Verification
Now our batch size has been updated from 10 to 20 we can verify that by re-running these commands to check the update of the block size

docker exec cli sh -c 'configtxlator proto_decode --input config_block.pb --type common.Block | jq .data.data[0].payload.data.config > config.json'
docker exec -e MAXBATCHSIZEPATH=$MAXBATCHSIZEPATH cli sh -c 'jq "$MAXBATCHSIZEPATH" config.json'

Verification-Modifying the Batch Size in Hyperledger Fabric

6. Summary

In this article, we saw that how we can update the batch size or batch timeout the running Hyperledger Fabric network, I know it is a bit tedious task but with the help of some shell scripting, we can do this in a couple of minutes. I have added one script updateBatchSize.sh that can do all of this we learned above. You can find the scripts here.

If you find this article helpful do hit the clap button and follow me for more such informative articles.

You can find me on Linkedin or stalk me on GitHub? If that’s too social for you, just drop a mail to adityaprakashjoshi1@gmail.com if you wish to talk tech with me.

Also, Check out my course on Hyperledger Fabric Deployment on MultiHost