How to assess whether a Proof of Concept really works?

When there is a new shiny technology in town, prancing around and making exorbitant claims, it is easy to dismiss it and pass it off. But when the technology sticks around, and a crowd starts gathering, maybe it’s time to check it out and see if does what it says it does. Furthermore, why not take it for a drive and prove the concepts for yourself? In this article we look at the relevance of PoC’s, how we can assess them, and what the next steps are after a successful PoC.

PoC stands for Proof of Concept and is popular in various industries where the approach is similar to “try before you buy”. They can sometimes be considered as a prototype or a pilot where the purpose is to demonstrate or verify that certain concepts or theories live up to expectations in the real world before full implementation. But why is this important?

Relevance

Quite often when a new piece of technology comes along, there is a glossy brochure but little or no proven history of its performance. A PoC is important because it requires much less investment to understand how the technology works and the benefits that come with it.

The single biggest advantage of a PoC is seeing the technology in action and, depending on the scope of the PoC, to provide greater opportunities to explore more about the technology: its benefits, its limitations, and also to assess its suitability. Having said this, there are certain ways to create a PoC to maximise effectiveness.

Scope

The first step when creating or requesting a PoC is to outline the scope and the requirements. The requirements should be easy, especially if the new technology is replacing an existing system because a baseline list of requirements should already be present. The scope is more challenging.

One on end of the PoC spectrum, a brief demonstration using dummy data in an external cloud or virtual environment may suffice. On the other end of the spectrum, a subset of real but depersonalised or anonymised data may be provided for the PoC within a specified environment and potentially integrated with existing non-production systems.

Although the latter scenario will require more time and resources, it provides a richer experience with a greater probability for a successful outcome. Perhaps the biggest challenge is scope creep where seemingly minor additions are made to the PoC, often as an afterthought. The risk here is that these changes may end up affecting the assessment criteria so this has to be carefully considered.

The most effective PoC usually contains one or two simple use cases relevant to the industry in question, operating on a subset of de-identified production data. This is because the purpose of a PoC is to make it as real as possible which will help in assessing the outcome.

Aren’t all blockchains public?

One misconception is that blockchains need to be public to be the most beneficial, since they are open, transparent and with a large number of nodes to be secure. Hearing, “we are a private or a government organisation and we don’t want a public blockchain so a blockchain PoC is not required” is not uncommon, but is not entirely true.

While public blockchains have their uses, permissioned blockchains are also finding their way into plenty of organisations. Here, a group of trusted parties can also implement a blockchain network, and this is quite often referred to as consortium network, where the main benefit is that all the parties are known and trusted. This means that less energy is required to fight against malicious and unknown participants in an open network. Quite often, performance is better as well.

How to access Proof of Concepts

A list of key successful criteria should be created to help determine if a PoC is successful or not. Take for example a blockchain PoC. Many often refer to performance or transactions per section (tx/s) as a key criteria. Permissioned blockchains may perform better than public blockchains but this is not the full picture. Other important considerations include:

• Security of the network
• Reliability of the network
• Ease of maintenance
• Availability of resources
• Ease of use for customer adoption

This list is unique to each situation but the key is that they must be clearly defined in order to assess whether the PoC met the requirements or not.

A PoC where existing metrics from an existing or similar system is available is the most beneficial because it can clearly help determine whether the PoC is successful. Who would want to use a new technology if the benefits are only marginally better than the status quo? The point here is the concept of “if it wasn’t measured, it can’t be compared” and this extends to the likes of usability and value generation because, at the end of the day, a decision needs to be made from the PoC of whether the technology should be incorporated into the organisation or not.

Internal champion

One aspect that cannot be overlooked is identifying an internal champion or (ideally) champions. This person drives the PoC process and helps to explain and educate the technology within an organisation. They also act as the conduit between all the various parties involved, such as the technical team, management, business analysts and the vendor, and eventually provides a summary report and recommendation to the executive team.

What are the next steps after a successful proof of concept?

Let’s assume that a PoC was undertaken, the requirements and scope were clearly defined and measured, and the outcome was a success. In other words, the PoC for a blockchain project met all of its objectives, what next?

During the scoping phase, if an agile approach was taken, user stories would have been created (of which one or two would be applied to the PoC). From here, assuming of course that the board or the executives give the green light to proceed on the grounds of a successful PoC demonstration, the beginnings of a fully-fledged project can start to take shape.

A more detailed scope and requirements analysis needs to be undertaken. This will help the formulation of a budget in order for funding to be secured. Resources, contracts, service level agreements and all the various project related bits and pieces need to be ironed out. The duration of the project can vary from a few months to a few years, but the best approach is to break down the project into manageable phases or waves.

Summary

In this article we looked at what a proof of concept is and why it is important when assessing new technologies. We also highlighted that the scope and requirements of a PoC are important because a it is not a fully-fledged project but just a small piece of work to prove the concept or idea.

The creation of key success metrics, which varies based on the project, provides the ability to assess the success or failure of it in order to determine next steps. Assuming a successful outcome, the next steps are to start formal project initiation and identify the scope and requirements in its entirety.

Proof of Concepts are a great way to make viability assessments very quickly and at low cost to help in the decision making process and should be part of every organisation’s toolbox.

Sean is a blockchain trainer, researcher and author and spends his free time building proof of concepts such as Ubering Energy on the blockchain or placing his land title on the blockchain.

Sean also contributes his time as an organiser of the Wellington Blockchain meetup, organizer of the Wellington Smart Contract meetup, Blockchain Association of NZ committee member and ISO/TC307 Blockchain & DLT NZ committee member.

Blockchain Business Review from Apla provides high-quality educational material from the world of blockchain to inform the business community of the competitive advantage that can be gained by integrating distributed ledger data storage within organizations. Our mission is to promote knowledge about blockchain and its uses in both the private and public sector and demonstrate the value of blockchain integration.