Hyperledger Fabric chaincode unit testing: smart contract test driven development (TDD)

How to fast and continuously check smart contract logic with unit tests: blockchain transactions, fired events and chaincode permissioning

Testing stage is a critical requirement for software quality assurance, doesn’t matter is this web application or a smart contract. Tests must be fast enough to run on every commit to repository. CCKit, programming toolkit for developing and testing Hyperledger Fabric Golang chaincode, enhances the development experience with extended version of MockStub for chaincode testing.

Steps in chaincode development process

A smart contract defines the different states of a business object and governs the processes that move the object between these different states. Smart contracts allows architects and smart contract developers to define the business processes and structure of data that are shared across different organisations collaborating in a blockchain network.

The job of a smart contract developer is to take an existing business process and express it as a code in a programming language. Steps of chaincode development:

• Define chaincode model — schema for state entries, input payload and events
• Define chaincode interface
• Implement chaincode instantiate method
• Implement chaincode methods with business logic
• Create tests

Test driven development (TDD) or Behavioral Driven Development (BDD), possibly, single way to develop smart contracts.

Chaincode (smart contract) testing

Tests must ensure that chaincode works as expected:

• particular input payload leads to particular business object state change
• particular (invalid) input payload leads to validation or other errors
• particular object state allow subset of state transitions (state machine)

Any software testing (chaincode or web application for example) may either be a manual or an automated process. Manual software testing is led by a team or individual who will manually operate a software product and ensure it behaves as expected. In case of chaincode tests you can manually invoke chaincode via peer cli tools.

Automated software testing is the practice of instrumenting input and output correctness checks for individual units of code. During automated testing, code are executed in a test environment with simulated input.

Running chaincode

Deploying chaincode to blockchain network isn’t the quickest thing in the world, there’s a lot of time that can be saved with testing. Also, more importantly, since blockchain is immutable and supposed to be secure because the code is on the network, we rather not leave flaws in our code.

During chaincode development and deploying to live network we can divide testing to multiple stage — fast stage, when testing only smart contract logic, and more complicated stage, when we do integration testing with live blockchain network, multiple peers, deployed on-chain code (smart contracts) and off-chain application, that uses SDK to connect with blockchain network peers.

Chaincode DEV mode

Deploying a Hyperledger Fabric blockchain network, chaincode installing and initializing, is quite complicated to set up and a long procedure. Time to re-install / upgrade the code of a smart contract can be reduced by using chaincode dev mode. Normally chaincode are started and maintained by peer. In “dev” mode, chaincode is built and started by the user. This mode is useful during chaincode development phase for rapid code/build/run/debug cycle turnaround. However, the process of updating the code will still be slow.

MockStub — mocked chaincode stub

Mocking is a unit testing phenomenon which helps to test objects in isolation by replacing dependent objects with complex behavior with test objects with pre-defined/simulated behavior. These test objects are called as Mock objects.

The shim package contains a MockStub implementation that wraps calls to a chaincode, simulating its behavior in the HLF peer environment. MockStubdoes not need to start multiple docker containers with peer, world state database, chaincodes and allows to get test results almost immediately.

True unit tests typically run extremely quickly, as there is no runtime infrastructure to set up.

MockStub essentially replaces the SDK and peer environment and allows to test chaincode without actually starting your blockchain network. It implements almost every function the actual stub does, but in memory.

MockStub from https://github.com/hyperledger/fabric/ repository includes implementation for most of shim.ChaincodeStubInterface function, but until current version of Hyperledger Fabric (1.4), the MockStub has not implemented some of the important methods such as GetCreator or method for work with private state range, for example. Since chaincode would use GetCreator method to get transaction creator certificate for access control, it's critical to be able to stub this method in order to completely unit-test chaincode.

CCKit MockStub

CCKit testing package contains:

• MockStub with implemented GetTransient and others methods and event subscription feature
• Test identity creation helpers
• Chaincode response expect helpers

Example: “Commercial Paper” chaincode

Scenario

Official hyperledger fabric documentation contain detailed chaincode example — Commercial Paper smart contract which defines the valid states for commercial paper, and the transaction logic that transition a paper from one state to another. We will test commercial paper extended chaincode example based on CCKit library with chaincode method routing and protobuf state.

We can represent the lifecycle of a commercial paper using a state transition diagram: commercial papers transition between issued, trading and redeemed states by means of the issue, buy and redeem transactions.

Requirements

To produce tests first we need to define requirements to tested application. Let’s start by listing our requirements for commercial paper chaincode:

• It should allow the issuer to issue commercial paper
• It should allow the participant to buy commercial paper
• It should allow the owner to redeem commercial paper

Chaincode interface functions described in file chaincode.go, so we can see all possible operations (transactions) with chaincode data:

Getting started

Before you begin, be sure to get CCKit:

git clone git@github.com:s7techlab/cckit.git

This will fetch and install the CCKit package with examples. After that we need to install the dependencies using command:

go mod vendor

Creating Test Suite

Testing in Go

Go has a built-in testing command called go test and a package testing which gives a minimal but complete testing experience. In our example we use Ginkgo - BDD-style Go testing framework, built on Go's testing package, and allows to write readable tests in an efficient manner. It is best paired with the Gomega matcher library, but is designed to be matcher-agnostic.

As with popular BDD frameworks in other languages, Ginkgo allows you to group tests in Describe and Context container blocks. Ginkgo provides the It and Specify blocks which can hold your assertions. It also comes with handy structural utilities such as BeforeSuite, AfterSuite, etc that allows you to separate test configuration from test creation, and improve code reuse.

Ginkgo also comes with support for writing asynchronous tests. This makes testing code that use channels with chaincode events as easy as testing synchronous code.

Package with tests

To write a new test suite, create a file whose name ends _test.go that contains the TestXxx functions, in our case will be cpaper_extended/chaincode_test.go

Using separate package with tests cpaper_extended_test instead of cpaper_extended allows us to respect the encapsulation of the chaincode package: your tests will need to import chaincode and access it from the outside. You cannot fiddle around with the internals, instead you focus on the exposed chaincode interface.

Import matchers and helpers

To get started, we need to import the matcher functionality from the Ginkgo testing package so we can use different comparison mechanisms like comparing response objects or status codes.

We import the ginkgo and gomega packages with the . namespace, so that we can use functions from these packages without the package prefix. This allows us to use Describe instead of ginkgo.Describe, and Equal instead of gomega.Equal.

Bootstrap

The call to RegisterFailHandler registers a handler, the Fail function from the Ginkgo package. This creates the coupling between Ginkgo and Gomega.

Test suite bootstrap example:

Test structure

This particular test specification can be written using Ginkgo as follows:

Implementing tests

Now we go in depth to see how to create test functions specifically for chaincode development using MockStub features.

Creating chaincode instance

Tests suite usually starts with creating a new instance of chaincode, or we can also instantiate a new chaincode instance before every test spec. This depends on how and what we want to test. In this example, we instantiate a global Commercial Paper chaincode that can be used in multiple test specs.

Test chaincode “Init" method

All chaincode invocation (via SDK to blockchain peer or to MockStub) resulted as peer.Response structure:

During tests we can check Response attribute:

• Status (error or success)
• Message string (contains error description)
• Payload contents (marshaled JSON or Protobuf)

CCKit testing package contains multiple helpers/wrappers on ginkgo expect functions.

Most frequently used helpers are:

• ResponseOk (response peer.Response) expects that peer response contains ok status code(200)
• ResponseError (response peer.Response) expects that peer response contains error status code (500). Optionally you can pass expected error substring.
• PayloadIs(response peer.Response, target interface{}) expects that peer response contains ok status code (200) and converts response to target type using CCKit convert package

For example we can simply test that Init method (invoked when the chaincode is initialised) returns successful status code:

Test the “Issue" method

We expect that invocation of issue chaincode method will result in:

• response with Ok status
• event IssueCommercialPaper is fired

In the test we can invoke issue method via MockStub, check response status and check chaincode event. Chaincode events can be received from chaincodeEventsChannel. The BeEquivalentTo method of the expect functionality comes in handy to compare the event payload.

This test will block until a response is received over the channel paperChaincode.ChaincodeEventsChannel (chaincode event). A deadlock or timeout is a common failure mode for tests like this. A straightforward pattern in such situation is to add a select statement at the bottom of the function and include a <-time.After(X) channel to specify a timeout. Ginkgo has this pattern built in. The body functions in all non-container blocks ( It , BeforeEache etc ) can take an optional done Done argument.

Done is a chan interface{}. When Ginkgo detects that the done Done argument has been requested it runs the body function as a goroutine, wrapping it with the necessary logic to apply a timeout assertion. You must either close the done channel or send something (anything) to it to tell Ginkgo that your test has ended. If your test doesn't end after a timeout period, Ginkgo will fail the test and move on the next one.

The default timeout is 1 second. You can modify this timeout by passing a float64 (in seconds) after the body function. In this example we set the timeout to 0.1 second.

Test the “Get" method

We expect that invocation of get chaincode method will result in:

• response with Ok status
• response payload is marshaled *schema.CommercialPaper with attributes of commercial paper added at the previous step

PayloadIs allows to check response status and converts to *schema.CommercialPaper, then Expect helps to check equality of received data with expected values:

Test chaincode permissioning

Each user can have different permissions to work with chaincode methods. All permissioning is based on user certificates and Membership Service Provider Identifiers, which means that permissions always correspond to an X.509 certificate.

The simple car contains logic to control who can invoke carRegister method. Test use From MockStub method to set certificate and MSP id of invoker

Debugging chaincode execution

CCKit supports Hyperledger Fabric Chaincode Logger and its options, so you can use CORE_CHAINCODE_LOGGING_LEVEL environment variable. CCKit chaincode state wrapper outputs debug severity level messages, for example:

It shows there are several operations with chaincode state performed while chaincode execution:

• checked existence entry with OWNER key while handling init chaincode method
• put state entry with OWNER key
• put state entry with [_idx CommercialPaper ExternalId EXT0001] while handling issue chaincode method (no unique index forCommercial entry entity) ...

Running test

To run the test suite you have to simply run the command in the repository where the test suite is located:

if you have any failures in test you can use -ginkgo.failFast option to disable running additional tests after any test fails.

Conclusion

Chaincode MockStub is really useful as it allows a developer to test his chaincode without starting the network every time. This reduces development time as he can use a test driven development (TDD) approach where he doesn't need to start the network (this takes +- 40-80 seconds depending on the specs of the computer).