Or How I Learned to Stop Worrying and Love Hyperledger Fabric and Kubernetes
Working with a bleeding edge technology is not for the faint of heart. This counts double when you are working with two bleeding-edge technologies. And when one of these technologies is Blockchain, well…
Blockchain applications however have a set of challenges that make them more difficult to deploy than other simpler technologies. Most front-ends, back-ends and databases are relatively easy to deploy, because all you need is one or more machines with some storage (whether physical or virtual) and access to the Internet. Such applications require relatively little coordination of resources, beyond providing a way to load-balance requests to them.
But Blockchain applications require considerably more complex deployment pipelines requiring, for example:
- initialising the blockchain genesis block,
- coordinating traffic between the different nodes (and potentially node types),
- deploying versioned smart contracts to them,
- managing tough security challenges, such as managing identities and cryptographic material of different components.
Our Blockchain framework of choice is Hyperledger Fabric, which is pretty cool, since it allows you to build permissioned blockchain networks that are quite flexible in what they can do, and can handle privacy (via collections) and sub-networks (via channels) quite elegantly.
However, deploying a Hyperledger Fabric network is a pain in the… neck. While the build your first network (byfn) example is a good start, it’s just that, a start. Long story short, you can’t build a reliable blockchain network on a single computer with bash scripts. At least, we don’t, because we want to sleep at night.
For one, we would like to deploy a blockchain network on a network of computers, so we used Kubernetes (K8S). Because we did not want to be handling a bunch of K8S manifests, we built 4 Helm charts, that you can peruse here. These allowed us to abstract each Hyperledger Fabric node into an abstract package we could easily deploy (and destroy, if needed).
We used these very Helm charts to deploy our (now well-known) project in Tanzania, where we recorded the journey of pregnant women on the blockchain, leading to the first-ever recorded birth of a baby on the blockchain: https://www.engadget.com/2018/07/27/first-blockchain-baby-charity/
But since deploying each of these components was only a sub-part of deploying a Blockchain network, we created some tutorials to explain the deployment of a basic Fabric network end-to-end.
This led us to release a webinar with the Linux Foundation in November 2018, which you can find via the blogpost below:
Have you always wanted to know how to deploy Hyperledger Fabric on Kubernetes? Then we have something just for you! We…www.hyperledger.org
Not content with the short webinar, we also presented a longer and more detailed workshop at the Hyperledger Global Forum in Basel in December 2018. You can find the slides here:
Deploying Hyperledger Fabric to production on Kubernetes is not a solved topic, AID:Tech present their work on…hgf18.sched.com
And you can even watch the recording of the first iteration of the workshop on the Friday the 14th of December.
The workshop showed us that this was a pain point, since the first time we gave it we could barely seat everyone in the room. But the confused looks after showing them the dozens of commands necessary to spin up a cluster led us to understand that we needed something easier to use, something more automated. So on December the 17th (yes, 3 days after the workshop), we started releasing some of our internal code for auto-magically deploying Hyperledger Fabric networks.
Now, we are big fans of test-driven development (TDD), and code reusability (which are difficult, though I admit not impossible, to achieve using Bash), so instead of a set of scripts, we developed a fully-fledged, unit- and integration-tested library called Nephos (meaning cloud in greek), and eventually added documentation (because, let’s face it, nobody uses a tool they can’t understand). By the time you read this, we will likely be a Hyperledger labs project.
We presented this to the world (well, to Dublin, which is the centre of our AID:Tech world) on February the 27th, 2019.
Above you can find a video of our tech presentation, where I describe using Nephos for a QA example of a super-simple network, but we have more complex examples in the repo and are continually working on improving it.
If you dig open-source, cloud and Blockchain, give it a spin, and feel free to contribute a PR to our open source repo at:
Library to deploy Hyperledger Fabric projects to a Kubernetes cloud - aidtechnology/nephosgithub.com