Hyperledger fabric cluster with multiple orderers and kafka

Production ready hyperledger fabric cluster

(λx.x)eranga

Published in

Coinmonks

6 min readJan 16, 2019

Happyledger :)

When you get in to hyperldger setup you will see it’s not a happy setup at all. Lot of configurations, lot of deployments files, simply it’s a lot of struggle :). When I’m get in to kubernetes I thought it is the most hardest thing to understand and configure. But when I’m working with hyperledger I felt its wrong :). So I thought to write some posts about in and out details of hyperldger fabric setup.

Discover and review best Blockchain softwares

I will write series of posts about hyperldger fabric. I named this blog serias as Happyledger :) Following are the topics that I’m gonna cover with Happyledger series.

Deploy production ready hyperldger cluster in single node env
Deploy production ready hyperldger cluster on multi node env
Deploy production ready hyperldger cluster on kubernetes
Write scala based hyperledger client application
Writing chaincode and testing them on local env

In this post I will cover how to deploy production ready hlf cluster(with multiple peers, multiple ordereres and kafka). All the deployment files and configurations are at hlf-deployment gitlab repo. Please checkout the repository and follow the post.

Service structure

The hyperledger network that I’m gonna setup contains following services. In this post these services will be deployed in a single host. In next post I will extend this setup for multi host environment.

One organization (org1.example.com)
Three peers (peer0.org1.example.com, peer1.org1.example.com, peer2.org1.example.com)
One certificate authority (ca.example.com)
Three orederers (orderer0.example.com, orderer1.example.com, orderer2.example.com)
Three zookeeper nodes (zookeeper0, zookeeper1, zookeeper2)
Four kafka nodes (kafka0, kafka1, kafka2, kafka3)

Following are the steps to configure and deploy the cluster.

1. Generate configs

1.1. Setup env

In order to work with hlf we need to have crypto certificates, genesis block and channel configurations. To generate them we are using scripts on bin directory and configurations(crypto-config.yaml and configtx.yaml) on config directory. So we need to add bin directory $PATH and set the FABRIC_CFG_PATH with config directory (where configxtx.yaml exists).

1.2. Generate crypto materials

Now we need to generate certificates/keys for our orderer, peers and ca. We are using cryptogen script to generate them. This script will generates crypto materials in crypto directory.

1.3. Generate genesis block

Next we need to generate genesis block for orderer. It can be done by using configtxgen script.

1.4. Generate channel config transaction

Now we need to generate channel configuration transaction by using configtxgen script.

1.5. Generate anchor peer transaction

Finally we need to generate anchor peer transactions for each organization by using configtxgen script. In this scenario we have only one organization (org1). If there are multiple organizations we need to do this step for all orgs(ex org1, org2 etc).

Now our configurations are ready. Next thing is to deploy the docker containers and running the services.

2. Deploy services

2.1. Define CA certificate/key

Before start the services, we need to define the CA certificate and key in deployment/docker-compose-kafka.yaml. CA certificate and key can be found in crypto-config/peerOrganizations/org1.example.com/ca directory. Define CA certificate with FABRIC_CA_SERVER_CA_CERTFILE variable and CA key with FABRIC_CA_SERVER_CA_KEYFILE variable like below.

2.2. Start services

Now we can deploy the docker containers which relates to fabric services and cli service. One thing to notice on docker-compose-kafka.yaml is, in each peer it defines CORE_PEER_GOSSIP_BOOTSTRAP address. In fabric network orderer only delivers blocks to single peer in each organization(leader peer). Through a process call gossip other peers in the network receive the block. More information about gossip in fabric can be found from here.

CORE_PEER_GOSSIP_BOOTSTRAP is used to bootstrap gossip within an organization. If you are using gossip, you will typically configure all the peers in your org to point to an initial set of peers for bootstrap (you can specific a space-separated list of peers). In here I have used one peer as the gossip bootstrap address. For an example in peer1 and peer2, I have used peer0 as the gossip bootstrap. In peer0 I have used peer1 as the gossip bootstrap.

In fabric, initial cross organization bootstrap information provide via the anchor peer. That means, initially a peer in an organization will only know about the anchor peers. To inform the gossip end point of a peer to other peers in different organization you need to set the CORE_PEER_GOSSIP_EXTERNALENDPOINT. If this is not set, then the endpoint information about the peer will not be broadcast to peers in other organizations. If simply says, in order to gossip with peers in other organizations you need to set CORE_PEER_GOSSIP_EXTERNALENDPOINT.

It will start following container on your machine.

One ca container
Three orderer containers
Three peer containers
Three zookeeper containers
Four kafka containers
One cli container

3. Setup channel

3.1. Create channel

First we need to create a channel with previously created channel configurations transaction. In here we are connecting to peer0.org1.example.com docker container and execute channel create command on it.

Next we need to join all of our peers(3 peers) into this channel.

3.2. Join peer0 to channel

Now we are connecting to peer0.org1.example.com docker container and execute channel join command on it. This command will generates mychannel.block inside the peer0.org1.example.com docker container.

3.3. Copy mychannel.block to other peers

As mentioned about when join peero.org1.example.com peer to channel it generate mychannel.block. We need to copy this file into peer1.org1.example.com and peer2.org1.example.com containers. Then only these contains can join to the the channel. I’m use docker cp command here to copy the file.

We can use cli container to create channel and join peers to channel as well. If you are using it you don’t need to copy the files and do above steps. Please follow this article for more information.

3.4. Join peer1 to channel

Now we can join peer1.org1.example.com to the channel.

3.5. Join peer2 to channel

As same as pervious, I can join peer2.org1.example.com to the channel.

4. Setup chaincode

Now we need to install chaincode on all the peers. Chaincode resides on chaincode directory. We are using cli container to install the chaincode. Cli container connects to one of the peer(peer0, peer1, peer2) to work. Connecting peer defines with CORE_PEER_ADDRESS env variable in cli container. When installing the chaincode in a peer, hlf will starts a separate docker container to run the chaincode.

4.1. Install on peer0

In order to install chaincode on peer0, cli container we need to connect to peer0. Set CORE_PEER_ADDRESS=peer0.org1.example.com:7051 in docker-compose-cli.yaml and recreate the cli. Then execute below command.

4.2. Install on peer1

Set CORE_PEER_ADDRESS=peer1.org1.example.com:7051 and execute below command.

4.3. Install on peer2

Set CORE_PEER_ADDRESS=peer2.org1.example.com:7051 and execute below command.

4.5. Instantiate chaincode

Now we have installed chaincode on all the peers. Next thing is to instantiate the chaincode. Chaincodes are instantiating only one time on the channel(not in peers).

Now our network is ready and chaincode is up and running on each peer. We can do transaction with the running chaincode.

5. Do transactions

There are two types of transactions in hyperldger, invoke and query. Following is the way to do the transactions.

5.1. Invoke transaction

With invoke, chaincode can modify the state of the variables in ledger. Each invoke transaction will be added to the ledge (it update ledger state). You can connect to any peer and do invoke/query transactions. In here I’m connecting to peer0(with cli container) and executed invoke transaction.

5.2. Query transaction

With query, chaincode will read the current state and send it back to the user. This transaction is not added to the ledger (it not update ledger state). Now I’m executing query by connecting to peer2

Whats next

Now you have setup multi peer, multi orderer hyperledger cluster with kafka on single host environment and execute transactions on it. We have used cli container to execute the transactions. In next post I’m gonna show how to deploy fabric network in multi host environment. Happy reading :)

Reference

Get Best Software Deals Directly In Your Inbox