Quality assurance for blockchain applications

According to few surveys, only 1% of blockchain projects have reached to production and 92% projects have failed due to multiple reasons. One of the most prominent reason among is lack of QA standards and knowledge for blockchain projects. After working on many pilot solutions with blockchain for different business cases, I can share my thought on “Quality of QA and tools needed for blockchain projects”. Focus of this article is to guide testers in the right direction so that they can understand the concept and start looking for blockchain projects for testing. I will write my next article to elaborate more on the techniques and frameworks needed.

First question that comes in every tester’s mind that how does one go about testing this new technology, and what are the limitations they have to deal with? Are they need to equip themselves with new testing techniques and tools? In my opinion, they just need critical thinking around what they have done in past and need to learn very few specialized techniques to deal with Blockchain projects.

Ideally, Blockchain technology is set to bring breakthrough changes in the way businesses are conducted today through its promise of secure, immutable, peer-to-peer decentralized trust networks with distributed consensus, shared, distributed ledger and smart contracts. These very features have added additional dimensions and standpoints for businesses that need special attention to certify quality assurance. Let’s analyze the different validation and verification methods and pointers we need to be understood before starting testing a blockchain-based applications. Please keep in mind that few of the terminologies that I have used below is more close to ethereum but the basic idea is applicable to any blockchain technology.

Ethical responsibilities — A solemn promise from tester needed because they know they are going to test a system that brings trust, where data is immutable and millions of dollars are at stake. :)

Knowledge of Blockchain — QA (testers) should have knowledge of many blockchain technologies and frameworks. They should understand the basic difference among different cryptographic algorithms (data encrypted and decrypted) and consensus protocols (impact on performance and latency).

Shifting left development environment — QA engineer should be an integral part of the team who develops the user stories and requirements. They should be writing agile test plans and BDD scenarios that can be later used for acceptance tests.

P2P Network Testing — Need to prepare a framework or test plan that considers the three-dimensional aspects of P2P systems: functionality, scalability, and volatility. Apart from that, we need to prepare test cases/plans for following components of a network:

1. Nodes:

· All nodes are up and running.

· Data should be the same at all the nodes after propagation of data.

· Nodes sending data simultaneously to the other nodes.

· Can add new node to the network.

· Can remove node from the network.

· Shared ledger same at each and every node after data diffusion.

· Verifying Address of node.

· Get the address/secret keys from all the nodes and save it.

2. Data Transmission in network:

· All node receives the data and if the data store is updated in chronological order.

· Hack data of one node and see whether data transmission effect other nodes as well. It should not.

· Start propagating data from two more nodes at the same time and verify the data store of all the other nodes.

3. Security and Fault tolerance of network:

· Try DDos attack and prevention techniques.

· Simulating Byzantine nodes.

· Resiliency test by validating the ledger after nodes that restarted or rejoined the network sync with other validating peers.

4. Test Network Types:

· Public — Anybody can access the ledger and see transactions.

· Private — Only selected parties are able to access the ledger and see transactions.

· Write/Commit access — Refers to who can take part in making changes to a distributed ledger (e.g., who can add blocks to a blockchain).

· Permissionless (open): Anyone can, in theory, participate in the consensus process (in practice, however, often limited by resource requirements such as owning suitable hardware or cryptocurrency).

· Permission (closed): Only selected parties can make changes to the distributed ledger.

5. Test on-chain Data: Process or transaction that takes place directly on the distributed ledger network.

6. Test off-chain Data: Process or transaction that is external to the distributed ledger.

7. Test tokenization: Refers to the process of digitally representing an existing, off-chain asset on a distributed ledger.

8. Test data diffusion: Refers to how and to whom data is broadcast in a distributed ledger network.

· Global: Data is broadcast to every network participant

· Multi-channel: Data is only broadcast to counterparties involved in a specific trade (selective disclosure).

9. Monitoring gas cost — To execute code logic on ethereum virtual machine we need to spend gas cost. QA team can define the strategies to monitor gas cost to identify bad code scenarios.

Test Smart Contract — Dynamically changing and expanding nature of the blockchain network makes testing of smart contract scenarios very hard. Testing smart contracts, involves simulation of all possible expected and unexpected conditions for every contract, testing all permutation and combinations of business logic and the proper triggering and correct execution of transactions and events.

As we know, smart contracts cannot consume/call third-party API directly. Therefore, testers have to have knowledge of oraclize like tools to test the consistency of responses of third-party systems using it.

Test Integration with data lake like IPFS — Blockchain applications are usually using IPFS like data lake to store data and only save hashes to blockchain ledgers. Tester has to understand the way to communicate to data lake systems.

Nonfunctional Testing — Performance and latency can vary based on network size and transaction load. Consensus algorithm is also putting a great impact on performance. Testing for multi-layered security system on the network and privacy of data in the ledger based on selective permissions is also required.

That’s it for this article. I will try to put more information on how current practices of QA can be implemented and extended for blockchain application test. Also will write about techniques and frameworks like a truffle to test ethereum based app.

This story is published in The Startup, Medium’s largest entrepreneurship publication followed by +406,714 people.

Subscribe to receive our top stories here.