Updating the Consortium Definition in Hyperledger Fabric

Hyperledger Fabric (HLF) is a permissioned blockchain platform designed for enterprise use. Privacy and confidentiality, forefront requirements of any enterprise solution, are provided through segregation of the network into channels, allowing groups of participants to transact on a private ledger. The creation of channels is controlled by members of Consortia, which consist of one or more organizations that are defined at the network level. As Fabric networks evolve and grow, it is expected that the list of organizations requiring the ability to create channels will change. Therefore, we need the ability to add or modify Consortia definitions without interrupting any of the network components. This article provides a guide to updating Consortia definitions in a step-by-step manner.

The following is based on the Adding an Org to a Channel tutorial in the Hyperledger Fabric documentation. In the tutorial, the user is guided through adding an organization as a channel administrator despite not being included in the Consortium definition used to create the channel. This is done by retrieving the latest configuration transaction for a channel, modifying it with the new organization details and submitting a new transaction to the channel to update the configuration. This process gives an organization the same privileges as organizations that were a part of the Consortium definition used to create the channel, but only for that one particular channel. If the organization was not defined as a part of a Consortium definition during network bootstrapping it will not be able to create a new channel.

The process of updating Consortia definitions to allow new groups of organizations to create channels together follows the same procedure of updating a channel configuration. It requires fetching and modifying the latest configuration transaction in the ordering service channel, and submitting the new transaction to update the network configuration. It is expected that the reader has some familiarity with Hyperledger Fabric concepts, such as the Ordering Service, peers, and how identity is managed with Membership Service Providers and certificate authorities.

Prerequisites

If you wish to follow along, please ensure that all prerequisites for running a HLF network have been installed on your computer. These can be found in the HLF documentation here and here. Note that Fabric version 1.4 was used in this article.

Because this tutorial involves processing JSON through the command line, a tool called jq is also required to be installed. For OS X users, jq can be installed with Homebrew with the command brew install jq. For Ubuntu users, you can use sudo apt-get install jq. For other operating systems, please refer to the official documentation for installation instructions.

The example code can be cloned from the GitHub repository. This example project has some pre-built scripts numbered 1–8 in the order that they should be executed. Script 4-add-org-to-channel is optional, and automates the process of adding an organization to a channel that is described in the Hyperledger Fabric Tutorial. It serves as a useful comparison to the 5-add-org-to-consortium script, which is the purpose of this article and is described in detail below.

It is also possible to modify the steps with your own Fabric network, making sure to update variables appropriately (e.g. organization names, channel names, etc.). If this is the case, skip to the updating the ordering channel configuration section.

The base network

In the example project, the base network consists of:

• an ordering service with a single node running solo consensus
• a single organization with a single peer
• a certificate authority (CA) for managing the cryptographic material for the organization
• a couchdb container to store the ledger for the organization
• a cli container for accessing the peer node
• a cli container for accessing the orderer node

Generate the crypto material and create the channel configuration transactions

Before the network can be started, the cryptographic material for the organization needs to be generated. This is done with the cryptogen command, which takes in the configuration file crypto-config.yaml and generates the certificates for each of the network components in the crypto-config folder.

The channel configuration transactions are created with the configtxgen command, which uses the configtx.yaml file specified in the FABRIC_CFG_PATH. Of note in the configtx.yaml file are the definitions of the organizations that comprise the network, as well as profiles containing the Consortia definitions that are used during channel creation. Configuration transactions are created for the orderer channel, the channel for the organization, and for registering the anchor peer in the organization channel. These transactions are stored in the config folder.

To generate all of these materials, make sure that you are in the fabric-dynamic-network directory, then run the generate script with:

./1-generate.sh

In the terminal window you should see logs similar to the following.

Output on successful completion of the generate script

Start the network

The containers with the Fabric components are brought up with the docker-compose command that uses the docker-compose.yaml file.

The organization is then able to create the channel with the channel configuration block, and then join the channel using the peer command.
This has been automated in the start script:

./2-start.sh

You should see the list of docker containers being created, then logs of the channel creation transactions being submitted, as per the below screenshot.

Output on successful completion of the start script

At any stage, it is possible to verify that the containers are still running using the command docker ps -a.

Stop the network

If you wish to bring down the network at any stage, you can use the stop script(./7-stop.sh) to stop the containers and the teardown script (./8-teardown.sh) to remove the containers and their images. If you do bring the network down, you will be required to rerun both the 1-generate.sh and 2-start.sh scripts to restart the network.

Create a new organization

The material needed for creating a new organization, hard-coded as Org2, can be found in the org2-artifacts folder. This consists of a crypto-config.yaml file used to define the cryptographic material that needs to be generated and the configtx.yaml to define the organization configuration and the new channel profile. Run the script 3-create-new-org.sh to generate all the cryptographic material required for the new organization and start the following containers for the Fabric components:

• a single peer container for the organization
• a certificate authority container for managing the cryptographic material for the organization
• a couchdb container to store the ledger for the organization
• a cli container for accessing the peer node

While the script is running, you should see the Org2 containers starting.
The components for the organization should be running and accessible at this point, which can be verified with the docker ps -a command.

Output on successful completion of the create-new-org script

However, because the organization was not defined in any Consortium in the original configuration block (the genesis block) for the ordering service, the new organization will not be permitted to interact with any of the existing channels or create a new channel.

Any attempt to fetch a block from an existing channel will be met with the error Error: can't read the block: &{FORBIDDEN}. This can be verified by running the command docker exec cli2 peer channel fetch 0 block_0.pb -o orderer.example.com:7050 -c channel1.

If the new organization attempts to create a channel with a Consortium that is not defined in the ordering channel it will also result in an error. This can be tested with the script 6-create-new-channel.sh, which should result in the error BAD_REQUEST-Unknown consortium name: SampleConsortium2.

In order for the new organization to create a new channel, it needs to be added to a Consortium definition in an updated configuration block in the ordering service channel.

Update the ordering channel configuration

We are now ready to update the ordering channel configuration to allow the new organization to create its own channels. The entire process has been automated in the script 5-add-org-to-consortium.sh, but the steps will be described in detail in the section below.

To follow along, this section assumes that there is an existing Fabric network running and that the new organization has cryptographic material generated and accessible by the cli container for the organization and the ordering service.

The new organization should have a file named configtx.yaml that specifies the policies associated with the new organization. In the example project, the configtx.yaml file for Organization 2 is in the org2-artifacts folder and contains the following:

org2-artifacts/configtx.yaml

Organizations:
 — &Org2
 Name: Org2MSP
 ID: Org2MSP
 MSPDir: crypto-config/peerOrganizations/org2.example.com/msp
 Policies:
 Readers:
 Type: Signature
 Rule: “OR(‘Org2MSP.member’)”
 Writers:
 Type: Signature
 Rule: “OR(‘Org2MSP.member’)”
 Admins:
 Type: Signature
 Rule: “OR(‘Org2MSP.admin’)”
 Endorsement:
 Type: Signature
 Rule: “OR(‘Org2MSP.member’)”
 AnchorPeers:
 — Host: peer0.org2.example.com
 Port: 8051

Prepare the new material to be added to the ordering service channel configuration transaction

The new channel configuration for the ordering service will need to contain the new Consortium definition. This information is currently specified in the configtx.yaml file and will need to be converted to a form that can be added directly into a configuration transaction. To do this, the configtxgen tool can be used. The configtxgen tool takes in the configtx.yaml file that is specified at the FABRIC_CFG_PATH, so we will define the path to the new org config file first.

In this example project, this can be done with:

cd org2-artifacts
export FABRIC_CFG_PATH=${PWD}
configtxgen -printOrg Org2MSP > ../crypto-config/org2.json

This should create a file called org2.json in the fabric-dynamic-network/crypto-config folder. This file contains the configuration of the new organization, including policy definitions and certificates for the administrator and CA root.

Prepare a new configuration block for the ordering service channel

Creating a new configuration block first requires retrieving the current configuration block, appending the new information to the block, creating an update transaction containing the delta between the original and the new configuration transaction, and then wrapping the update transaction with a header to make it recognizable to the ordering service.

To do this, the configuration tool configtxlator is used. At various stages the configtxlator tool requires that the data is represented in JSON format or as a protobuf object, meaning that there are several conversion steps going from JSON to protobuf and vice versa. This makes the process of creating the updated transaction somewhat tedious, but please bear with me through it!

As this process requires manipulation of JSON objects through the command line, a tool called jq is also used.

1. Fetch the latest configuration block for the ordering service channel as the orderer cli.

docker exec cliOrderer peer channel fetch config sys_config_block.pb -o orderer.example.com:7050 -c ordererchannel

If successful, the final message printed in the terminal should include Received block: 0, as per the below screenshot.

Output on successful completion of the fetch config command

2. Convert the configuration transaction in the block from protobuf to JSON and trim it down using the jq tool.

docker exec cliOrderer configtxlator proto_decode --input sys_config_block.pb --type common.Block | jq .data.data[0].payload.data.config > ../crypto-config/sys_config.json

This creates a new file in the fabric-dynamic-network/crypto-config folder called sys_config.json that will contain the current network configuration, including the existing SampleConsortium1.

3. Append the new Consortium definition to the configuration file.

docker exec cliOrderer jq -s ‘.[0] * {“channel_group”:{“groups”:{“Consortiums”:{“groups”: {“SampleConsortium’2'”: {“groups”: {“‘Org2’MSP”:.[1]}, “mod_policy”: “/Channel/Orderer/Admins”, “policies”: {}, “values”: {“ChannelCreationPolicy”: {“mod_policy”: “/Channel/Orderer/Admins”,”value”: {“type”: 3,”value”: {“rule”: “ANY”,”sub_policy”: “Admins”}},”version”: “0”}},”version”: “0”}}}}}}’ ./crypto/sys_config.json ./crypto/org2.json > ../crypto-config/modified_config.json

This modified_config.json file contains the current configuration information as well as the new SampleConsortium2 definition that the channel needs to be updated with. As the update command only requires the delta information, we still need to create a transaction with just the difference between the original sys_config.json and the modified_config.json.

4. Convert the original configuration JSON file to protobuf.

docker exec cliOrderer configtxlator proto_encode --input ./crypto/sys_config.json --type common.Config --output ./crypto/sys_config.pb

A file called sys_config.pb should be created in the fabric-dynamic-network/crypto-config folder.

5. Convert the new configuration JSON file to protobuf.

docker exec cliOrderer configtxlator proto_encode --input ./crypto/modified_config.json --type common.Config --output ./crypto/modified_config.pb

A file called modified_config.pb should be created in the fabric-dynamic-network/crypto-config folder.

6. Create a protobuf with the difference between the two configs

docker exec cliOrderer configtxlator compute_update --channel_id ordererchannel --original ./crypto/sys_config.pb --updated ./crypto/modified_config.pb --output ./crypto/org2_update.pb

A file called org2_update.pb should be created in the fabric-dynamic-network/crypto-config folder, which contains the update configuration.
This needs to be wrapped with the appropriate tags and header in order to be properly processed by the peer update command. To do this the protobuf version needs to be converted to JSON, the wrappers added, and then converted back to protobuf to submit to the orderer.

7. Convert the delta config protobuf file to JSON

docker exec cliOrderer configtxlator proto_decode --input ./crypto/org2_update.pb --type common.ConfigUpdate | jq . > ../crypto-config/org2_update.json

The org2_update.json file created should contain both the SampleConsortium1 and SampleConsortium2 definitions in the write set section of the JSON.

8. Add the header field to the update configuration JSON file.

docker exec cliOrderer echo ‘{“payload”:{“header”:{“channel_header”:{“channel_id”:”ordererchannel”, “type”:2}},”data”:{“config_update”:’$(cat ../crypto-config/org2_update.json)’}}}’ | jq . > ../crypto-config/org2_update_in_envelope.json

This should have created an org2_update_in_envelope.json file that includes a “payload” section and wraps the configuration in a “data” section.

9. Convert the JSON back to protobuf.

docker exec cliOrderer configtxlator proto_encode --input ./crypto/org2_update_in_envelope.json --type common.Envelope --output ./crypto/org2_update_in_envelope.pb

The org2_update_in_envelope.pb file that should have been created is now ready to be submitted to the orderer to be added to the ordering service channel blockchain.

10. Submit update configuration transaction.

docker exec cliOrderer peer channel update -f ./crypto/org2_update_in_envelope.pb -c ordererchannel -o orderer.example.com:7050

If successful, the final message in terminal should be Successfully submitted channel update, similar to the following screenshot.

Output on successful completion of the peer channel update command

Verify that the new organization is able to create a new channel

At this point, the new organization should be able to create a new channel using the new Consortium definition. To verify this, ensure the configtx.yaml in the org2_artifacts folder includes a channel profile with the Consortium that Org2 is a member of. An example file is shown below.

org2-artifacts/configtx.yaml

Organizations:
 — &Org2
 Name: Org2MSP
 ID: Org2MSP
 MSPDir: crypto-config/peerOrganizations/org2.example.com/msp
 Policies:
 Readers:
 Type: Signature
 Rule: “OR(‘Org2MSP.member’)”
 Writers:
 Type: Signature
 Rule: “OR(‘Org2MSP.member’)”
 Admins:
 Type: Signature
 Rule: “OR(‘Org2MSP.admin’)”
 Endorsement:
 Type: Signature
 Rule: “OR(‘Org2MSP.member’)”
 AnchorPeers:
 — Host: peer0.org2.example.com
 Port: 8051
Application: &ApplicationDefaults
 Organizations:
 Policies:
 Readers:
 Type: ImplicitMeta
 Rule: “ANY Readers”
 Writers:
 Type: ImplicitMeta
 Rule: “ANY Writers”
 Admins:
 Type: ImplicitMeta
 Rule: “MAJORITY Admins”
 LifecycleEndorsement:
 Type: ImplicitMeta
 Rule: “MAJORITY Endorsement”
 Endorsement:
 Type: ImplicitMeta
 Rule: “MAJORITY Endorsement
Channel: &ChannelDefaults
 Policies:
 Readers:
 Type: ImplicitMeta
 Rule: “ANY Readers”
 Writers:
 Type: ImplicitMeta
 Rule: “ANY Writers”
 Admins:
 Type: ImplicitMeta
 Rule: “MAJORITY Admins”
Profiles:
Org2Channel:
 Consortium: SampleConsortium2
 <<: *ChannelDefaults
 Application:
 <<: *ApplicationDefaults
 Organizations:
 — *Org2

This configtx.yaml is used by the create new channel script, to create a new channel and join the peer of Org2 to the channel. Navigate back to the fabric-dynamic-network folder and run ./6-create-new-channel.sh. Note that this script produced an error when it was run before adding the new Consortium definition to the ordering service channel. It should now complete successfully, with the terminal logs concluding with:

Channels peers has joined:
channel2

Output on successful completion of the create new channel script

Conclusion

Hopefully this article has been helpful to describe how organizations can be added to a Consortium definition in order to grant them the ability to create and manage channels in an existing Hyperledger Fabric network. This enables new groups of organizations to control the creation of channels together, enabling Fabric networks to adapt to changes in privacy requirements as ecosystems grow.

References

- Hyperledger Fabric tutorials
- GitHub repository with example code

Acknowledgements

Thanks to Isabell Kiral, Lenin Mehedy and Dileban Karunamoorthy for their input and corrections to this post.