Migration from Kafka to RAFT in Hyperledegr Fabric 1.4.3

Download the fabric sample for here. Please visit the [installation instructions](link) to ensure you have the correct prerequisites installed.

Download the latest binaries and Docker images with 1.4.3. All installation instructions will utilize `scripts/bootstrap.sh`
If you did not find any script/bootstrap.sh` file,
# Fetch bootstrap.sh from fabric repository using the following command

curl -sS https://raw.githubusercontent.com/hyperledger/fabric/master/scripts/bootstrap.sh -o ./scripts/bootstrap.sh

# Change file mode to executable
chmod +x ./scripts/bootstrap.sh

Execute the following command from a fabric sample directory.

./scripts/bootstrap.sh

Default it will download the latest binaries and latest docker images if you want to download specific, execute the following command by giving specific version

./scripts/bootstrap.sh [version] [ca version] [thirdparty_version].

You can see all binaries are downloaded /bin folder.

Go to the first-network folder cd first-network

Before creating certificates please make the following changes in crypto-config.yaml the the file, comment the EnableNodeOUs: true and add CA as shown below(bold). For version 1.4.3, cryptogen tool is generating Node Organization unit (OU) as a client for orderer, For peer, it's creating correct as expected(Node ou as Admin).

Discover and review best Blockchain softwares

OrdererOrgs:
# ---------------------------------------------------------------------------
# Orderer
# ---------------------------------------------------------------------------
- Name: Orderer
Domain: example.com
# EnableNodeOUs: true
CA:
OrganizationalUnit: admin
# ---------------------------------------------------------------------------
# "Specs" - See PeerOrgs below for complete description
# ---------------------------------------------------------------------------
Specs:
- Hostname: orderer
- Hostname: orderer2
- Hostname: orderer3
- Hostname: orderer4
- Hostname: orderer5
Users:
Count: 1

Generate the certificate using the following command(default it uses cryptogen tool)

./byfn generate -o kafka

The next step is up the network using the following command. Here we are using state database as couch DB and ordering service node using Kafka zookeeper.

./byfn.sh up -o kafka -s couchdb

Once the network is up, you will be able to see the following success message.

You could see all containers are running by execauting command

`docker ps ` or docker container list

For any kind of configuration block change, we follow the same steps as below.

We need to follow following three steps for migration Kafka to RAFT

.1) Switch on Maintenance Mode
2) Change Consensus type to etcdraft
3) Switch Off Maintenance mode

STEP1

Switch ON Maintenace mode
Login to cli container

docker exec -ti cli bash

If we’re upgrading from v1.3 to v1.4.3, we need to set the system channel name to testchainid:

CHANNEL_NAME=testchainid

If we’re upgrading from v1.4.1 to v1.4.3, we need to set the system channel name to byfn-sys-channel:

CHANNEL_NAME=byfn-sys-channel

Note: Make sure that you are using the system channel name correctly

Execute the following commands for mychannel

# export all needed env vars
export ORDERER_CA=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem
export CHANNEL_NAME=mychannel # make sure channel name is correct
# change work directories
mkdir maintenance_on_$CHANNEL_NAME && cd maintenance_on_$CHANNEL_NAME
# fetch current channel config
peer channel fetch config config_block.pb -o orderer.example.com:7050 -c $CHANNEL_NAME --tls --cafile $ORDERER_CA
# decode fetched channel config
configtxlator proto_decode --input config_block.pb --type common.Block | jq .data.data[0].payload.data.config > config.json
# save old config, to calculate delta in the future
cp config.json config_mod.json
# set maintenance mode in configs
sed -i 's/NORMAL/MAINTENANCE/g' config_mod.json
# encode old config to protopuf
configtxlator proto_encode --input config.json --type common.Config --output config.pb
# encode new config to protopuf
configtxlator proto_encode --input config_mod.json --type common.Config --output modified_config.pb
# compute delta between configs
configtxlator compute_update --channel_id $CHANNEL_NAME --original config.pb --updated modified_config.pb --output config_update.pb
# decode delta config
configtxlator proto_decode --input config_update.pb --type common.ConfigUpdate | jq . > config_update.json
# wrap delta config with a header
echo '{"payload":{"header":{"channel_header":{"channel_id":"mychannel", "type":2}},"data":{"config_update":'$(cat config_update.json)'}}}' | jq . > config_update_envelope.json
# encode wrapped config to protopuf
configtxlator proto_encode --input config_update_envelope.json --type common.Envelope --output config_update_in_envelope.pb
# export all needed env vars
export CORE_PEER_LOCALMSPID="OrdererMSP"
export CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/tls/ca.crt
export CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/users/Admin@example.com/msp/
export CORE_PEER_ADDRESS=peer0.org1.example.com:7051
# submit new channel config
peer channel update -f config_update_in_envelope.pb -c $CHANNEL_NAME -o orderer.example.com:7050 --tls --cafile $ORDERER_CA

Expected output:

2019-11-26 07:15:08.563 UTC [channelCmd] InitCmdFactory -> INFO 001 Endorser and orderer connections initialized
2019-11-26 07:15:08.711 UTC [channelCmd] update -> INFO 002 Successfully submitted channel update

Do the same procedure for the system channel as well.

# export all needed env vars
export CHANNEL_NAME=byfn-sys-channel
# change work directories
cd ..
mkdir maintenance_on_$CHANNEL_NAME && cd maintenance_on_$CHANNEL_NAME
# fetch current channel config
peer channel fetch config config_block.pb -o orderer.example.com:7050 -c $CHANNEL_NAME --tls --cafile $ORDERER_CA
# decode fetched channel config
configtxlator proto_decode --input config_block.pb --type common.Block | jq .data.data[0].payload.data.config > config.json
# save old config, to calculate delta in the future
cp config.json config_mod.json
# set maintenance mode in configs
sed -i 's/NORMAL/MAINTENANCE/g' config_mod.json
# encode old config to protopuf
configtxlator proto_encode --input config.json --type common.Config --output config.pb
# encode new config to protopuf
configtxlator proto_encode --input config_mod.json --type common.Config --output modified_config.pb
# compute delta between configs
configtxlator compute_update --channel_id $CHANNEL_NAME --original config.pb --updated modified_config.pb --output config_update.pb
# decode delta config
configtxlator proto_decode --input config_update.pb --type common.ConfigUpdate | jq . > config_update.json
# wrap delta config with a header
echo '{"payload":{"header":{"channel_header":{"channel_id":"byfn-sys-channel", "type":2}},"data":{"config_update":'$(cat config_update.json)'}}}' | jq . > config_update_envelope.json
# encode wrapped config to protopuf
configtxlator proto_encode --input config_update_envelope.json --type common.Envelope --output config_update_in_envelope.pb
# sign channel update config
#peer channel signconfigtx -f config_update_in_envelope.pb
# export all needed env vars
export CORE_PEER_LOCALMSPID="OrdererMSP"
export CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/tls/ca.crt
export CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/users/Admin@example.com/msp/
export CORE_PEER_ADDRESS=peer0.org1.example.com:7051
# submit new channel config
peer channel update -f config_update_in_envelope.pb -c $CHANNEL_NAME -o orderer.example.com:7050 --tls --cafile $ORDERER_CA

Expected output

2019-11-26 07:22:58.069 UTC [channelCmd] InitCmdFactory -> INFO 001 Endorser and orderer connections initialized
2019-11-26 07:22:58.221 UTC [channelCmd] update -> INFO 002 Successfully submitted channel update

Restart all necessary container by running following command

docker restart $(docker ps -a | grep "hyperledger/fabric" | awk '{print $1}')

STEP 2
Change consensus type in configuration block

Login to cli container

docker exec -it cli bash

execute following commands

# export all needed env vars
export ORDERER_CA=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem
export CHANNEL_NAME=mychannel
export CORE_PEER_LOCALMSPID="OrdererMSP"
export CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/tls/ca.crt
export CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/users/Admin@example.com/msp/
export CORE_PEER_ADDRESS=peer0.org1.example.com:7051
# change work directories
mkdir switch_to_raft_${CHANNEL_NAME} && cd switch_to_raft_${CHANNEL_NAME}
# fetch current channel config
peer channel fetch config config_block.pb -o orderer.example.com:7050 -c $CHANNEL_NAME --tls --cafile $ORDERER_CA
# decode fetched channel config
configtxlator proto_decode --input config_block.pb --type common.Block | jq .data.data[0].payload.data.config > config.json
# save old config, to calculate delta in the future
cp config.json config_mod.json

Open Config_mod.json do the following update in this file. The file should be available in location workdir/switch_to_raft_mychannel

Replace the following block in config_mod.json file

"ConsensusType": {
"mod_policy": "Admins",
"value": {
"metadata": {
"consenters": [
{
"client_tls_cert": "LS0tLS1<…>LS0tLQo=",
"host": "orderer.example.com",
"port": 7050,
"server_tls_cert": "LS0tLS1<...>tLQo="
}
],
"options": {
"election_tick": 10,
"heartbeat_tick": 1,
"max_inflight_blocks": 5,
"snapshot_interval_size": 20971520,
"tick_interval": "500ms"
}
},
"state": "STATE_MAINTENANCE",
"type": "etcdraft"
},
"version": "1"
}

The client_tls_cert and server_tls_cert fields are actually equal and should contain the base64-encoded orderer service certificate. You can obtain this value by executing the following command in the CLI container we’ve already opened.

base64 /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/tls/server.crt -w0 && echo ""
# encode old config to protopuf
configtxlator proto_encode --input config.json --type common.Config --output config.pb
# encode new config to protopuf
configtxlator proto_encode --input config_mod.json --type common.Config --output modified_config.pb
# compute delta between configs
configtxlator compute_update --channel_id $CHANNEL_NAME --original config.pb --updated modified_config.pb --output config_update.pb
# decode delta config
configtxlator proto_decode --input config_update.pb --type common.ConfigUpdate | jq . > config_update.json
# wrap delta config with a header
echo '{"payload":{"header":{"channel_header":{"channel_id":"mychannel", "type":2}},"data":{"config_update":'$(cat config_update.json)'}}}' | jq . > config_update_envelope.json
# encode wrapped config to protobuf
configtxlator proto_encode --input config_update_envelope.json --type common.Envelope --output config_update_in_envelope.pb
# sign channel update config
# peer channel signconfigtx -f config_update_in_envelope.pb
# export all needed env vars
export CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/tls/ca.crt
export CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/users/Admin@example.com/msp/
export CORE_PEER_ADDRESS=peer0.org1.example.com:7051
# submit new channel config
peer channel update -f config_update_in_envelope.pb -c $CHANNEL_NAME -o orderer.example.com:7050 --tls --cafile $ORDERER_CA

Expected Output

2019-11-26 07:55:38.417 UTC [channelCmd] InitCmdFactory -> INFO 001 Endorser and orderer connections initialized
2019-11-26 07:55:38.572 UTC [channelCmd] update -> INFO 002 Successfully submitted channel update

Do the same above step for system channel

export CHANNEL_NAME=byfn-sys-channel
cd ..
# change work directories
mkdir switch_to_raft_${CHANNEL_NAME} && cd switch_to_raft_${CHANNEL_NAME}
# fetch current channel config
peer channel fetch config config_block.pb -o orderer.example.com:7050 -c $CHANNEL_NAME --tls --cafile $ORDERER_CA
# decode fetched channel config
configtxlator proto_decode --input config_block.pb --type common.Block | jq .data.data[0].payload.data.config > config.json
# save old config, to calculate delta in the future
cp config.json config_mod.json

Open Config_mod.json do the following update in this file. File should be available in location workdir/switch_to_raft_byfn-sys-channel

Replace the following block in config_mod.json file

"ConsensusType": {
"mod_policy": "Admins",
"value": {
"metadata": {
"consenters": [
{
"client_tls_cert": "LS0tLS1<…>LS0tLQo=",
"host": "orderer.example.com",
"port": 7050,
"server_tls_cert": "LS0tLS1<...>tLQo="
}
],
"options": {
"election_tick": 10,
"heartbeat_tick": 1,
"max_inflight_blocks": 5,
"snapshot_interval_size": 20971520,
"tick_interval": "500ms"
}
},
"state": "STATE_MAINTENANCE",
"type": "etcdraft"
},
"version": "1"
}

The client_tls_cert and server_tls_cert fields are actually equal and should contain the base64-encoded orderer service certificate. You can obtain this value by executing the following command in the CLI container we’ve already opened.

base64 /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/tls/server.crt -w0 && echo ""

Execute following on CLI

# encode old config to protopuf
configtxlator proto_encode --input config.json --type common.Config --output config.pb
# encode new config to protopuf
configtxlator proto_encode --input config_mod.json --type common.Config --output modified_config.pb
# compute delta between configs
configtxlator compute_update --channel_id $CHANNEL_NAME --original config.pb --updated modified_config.pb --output config_update.pb
# decode delta config
configtxlator proto_decode --input config_update.pb --type common.ConfigUpdate | jq . > config_update.json
# wrap delta config with a header
echo '{"payload":{"header":{"channel_header":{"channel_id":"byfn-sys-channel", "type":2}},"data":{"config_update":'$(cat config_update.json)'}}}' | jq . > config_update_envelope.json
# encode wrapped config to protopuf
configtxlator proto_encode --input config_update_envelope.json --type common.Envelope --output config_update_in_envelope.pb
# sign channel update config
# peer channel signconfigtx -f config_update_in_envelope.pb
# export all needed env vars
export CORE_PEER_LOCALMSPID="OrdererMSP"
export CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/tls/ca.crt
export CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/users/Admin@example.com/msp/
export CORE_PEER_ADDRESS=peer0.org1.example.com:7051
# submit new channel config
peer channel update -f config_update_in_envelope.pb -c $CHANNEL_NAME -o orderer.example.com:7050 --tls --cafile $ORDERER_CA

Expected Output

2019-11-26 07:55:38.417 UTC [channelCmd] InitCmdFactory -> INFO 001 Endorser and orderer connections initialized
2019-11-26 07:55:38.572 UTC [channelCmd] update -> INFO 002 Successfully submitted channel update

Exit CLI and restart all related to the fabric

docker restart $(docker ps -a | grep "hyperledger/fabric" | awk '{print $1}')

after 20 sec, check orderer log it should show like below

you can search with “etcdraft migration detected”, you will be able to see logs of raft ordering service.

STEP 3
Switch OFF maintenance Mode

Now we have updated the necessary configuration, we need to disable the MAINTENANCE mode to NORMAL mode for both channel(mychannel and sys-channel)

Login to CLIcontainer

docker exec -it cli bash

Run the following code in CLI

# export all needed env vars
export CORE_PEER_LOCALMSPID="OrdererMSP"
export CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/tls/ca.crt
export CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/users/Admin@example.com/msp/
export CORE_PEER_ADDRESS=peer0.org1.example.com:7051
export ORDERER_CA=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem
export CHANNEL_NAME=mychannel
# change working directory
mkdir maintenance_off_$CHANNEL_NAME && cd maintenance_off_$CHANNEL_NAME
# fetch current channel config
peer channel fetch config config_block.pb -o orderer.example.com:7050 -c $CHANNEL_NAME --tls --cafile $ORDERER_CA
# decode current config
configtxlator proto_decode --input config_block.pb --type common.Block | jq .data.data[0].payload.data.config > config.json
# save to perform changes in the configuration
cp config.json config_mod.json
# modify new config file
sed -i 's/MAINTENANCE/NORMAL/g' config_mod.json
# encode old config to protopuf
configtxlator proto_encode --input config.json --type common.Config --output config.pb
# encode new config to protopuf
configtxlator proto_encode --input config_mod.json --type common.Config --output modified_config.pb
# compute delta in configurations
configtxlator compute_update --channel_id $CHANNEL_NAME --original config.pb --updated modified_config.pb --output config_update.pb
# decode delta
configtxlator proto_decode --input config_update.pb --type common.ConfigUpdate | jq . > config_update.json
# wrap delta with header
echo '{"payload":{"header":{"channel_header":{"channel_id":"mychannel", "type":2}},"data":{"config_update":'$(cat config_update.json)'}}}' | jq . > config_update_envelope.json
# encode delta config
configtxlator proto_encode --input config_update_envelope.json --type common.Envelope --output config_update_in_envelope.pb
# sign config update transaction
# peer channel signconfigtx -f config_update_in_envelope.pb
# export needed env vars
export CORE_PEER_LOCALMSPID="OrdererMSP"
export CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/tls/ca.crt
export CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/users/Admin@example.com/msp/
export CORE_PEER_ADDRESS=peer0.org1.example.com:7051
# submit channel update
peer channel update -f config_update_in_envelope.pb -c $CHANNEL_NAME -o orderer.example.com:7050 --tls --cafile $ORDERER_CA

Repeat the same for sys-channel

export CHANNEL_NAME=byfn-sys-channel
cd ..
# change working directories
mkdir maintenance_on_$CHANNEL_NAME && cd maintenance_on_$CHANNEL_NAME
# fetch current channel config
peer channel fetch config config_block.pb -o orderer.example.com:7050 -c $CHANNEL_NAME --tls --cafile $ORDERER_CA
# decode fetched config
configtxlator proto_decode --input config_block.pb --type common.Block | jq .data.data[0].payload.data.config > config.json
# save config for further modifications
cp config.json config_mod.json
# modify channel config
sed -i 's/MAINTENANCE/NORMAL/g' config_mod.json
# encode old channel config
configtxlator proto_encode --input config.json --type common.Config --output config.pb
# encode new channel config
configtxlator proto_encode --input config_mod.json --type common.Config --output modified_config.pb
# compute delta between configuration
configtxlator compute_update --channel_id $CHANNEL_NAME --original config.pb --updated modified_config.pb --output config_update.pb
# decode delta
configtxlator proto_decode --input config_update.pb --type common.ConfigUpdate | jq . > config_update.json
# wrap delta with header
echo '{"payload":{"header":{"channel_header":{"channel_id":"byfn-sys-channel", "type":2}},"data":{"config_update":'$(cat config_update.json)'}}}' | jq . > config_update_envelope.json
# encode wrapped delta
configtxlator proto_encode --input config_update_envelope.json --type common.Envelope --output config_update_in_envelope.pb
# sign peer channel update
# peer channel signconfigtx -f config_update_in_envelope.pb
# export all needed env vars
export CORE_PEER_LOCALMSPID="OrdererMSP"
export CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/tls/ca.crt
export CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/users/Admin@example.com/msp/
export CORE_PEER_ADDRESS=peer0.org1.example.com:7051
# submit new channel config
peer channel update -f config_update_in_envelope.pb -c $CHANNEL_NAME -o orderer.example.com:7050 --tls --cafile $ORDERER_CA

Exit the CLI container and restart all containers with the following command

docker restart $(docker ps -a | grep "hyperledger/fabric" | awk '{print $1}')

For checking data persistence and invoking login into

docker exec -ti cli bash

Invoke and query chaincode with following commands

# export env vars
export ORDERER_CA=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem
export CHANNEL_NAME=mychannel
export PEER0_ORG1_CA=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt
export PEER0_ORG2_CA=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt
# invoke chaincode
peer chaincode invoke -o orderer.example.com:7050 -C $CHANNEL_NAME -n mycc --peerAddresses peer0.org1.example.com:7051 --tlsRootCertFiles $PEER0_ORG1_CA --peerAddresses peer0.org2.example.com:9051 --tlsRootCertFiles $PEER0_ORG2_CA -c '{"Args":["invoke","a","b","10"]}' --tls --cafile $ORDERER_CA
# query chaincode
peer chaincode invoke -o orderer.example.com:7050 -C $CHANNEL_NAME -n mycc --peerAddresses peer0.org1.example.com:7051 --tlsRootCertFiles $PEER0_ORG1_CA --peerAddresses peer0.org2.example.com:9051 --tlsRootCertFiles $PEER0_ORG2_CA -c '{"Args":["query","a"]}' --tls --cafile $ORDERER_CA

Get Best Software Deals Directly In Your Inbox