Getting Real with Blockchain
It’s pretty-much impossible to avoid the pronouncements in both the technical and mainstream press about how blockchain is going to change the world. Blockchain feels like the cure for every known ill in the world of computing. Take a couple of smart contracts three times a day and everything will be just fine. Haven’t we heard this sort of thing before, though? The details were different, but a myriad of other technologies promised equally radical futures. In far too many cases they were wrong, and the reality fell short of the hype. How, then, do we get real blockchain? More specifically, as I’m not pretending to be completely unbiased, how are we getting real with Corda?
Technology Hype Cycles
Technology developments tend to follow a common pattern, often known as the “hype cycle”. Figure 1 depicts journey. It should be pretty familiar to most of us.
Figure 1: Technology Hype Cycle (By Jeremykemp at English Wikipedia, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=10547051)
It’s not that early technology enthusiasts deliberately set out to oversell anything. They demonstrate how their particular technology can do amazing things, and how simple it is to use. Dig into the details, though, and we find far too many demonstrations are proofs-of-concept, or prototypes of solutions to oversimplified problems. The problem is the demos tend to look quite complete, so it’s far too easy for both their creators and their viewers to miss the size of the gap between them and production-viable solutions.
Production systems are much more complex. They have to address the full needs of a business, they have to integrate with other things, and they have to handle a whole range of failure modes. As a friend has often reminded me, “it’s easy to build things that work; it’s very much harder to build things that fail gracefully”. The transition to production, and thus to making the technology real, is often very painful. Early successes lead to an expectation that a technology is largely finished when, in practice, it’s often barely started at all. This trough of disillusionment can be very deep indeed.
This path to production is vital though. With understanding of production needs and production applications we start to see the emergence of useful software platforms.
What are Platforms?
In the world of software, we often talk about “platforms”. What do we mean, though?
Platforms seek to extract the core of a technology, allowing that core to be used to solve many different problems. They make the efforts of a group of very talented designers and engineers available to everyone else to build on. Platforms focus on things that are really hard to get right, allowing anyone using them to avoid worrying about those things too.
Great platforms go far further. Not only do they offer capabilities and guarantees to software built on top of them, but they go to great efforts to avoid having too many opinions about how that software should work. They enable success without constraining that success too much. In doing so, they allow for innovations that did not have to be imagined by the platform’s designers.
For a technology like blockchain we have to start with a great platform. If our platform has too narrow a scope it can’t hope to fulfil everyone’s aspirations. Equally though a platform has to make a few decisions otherwise it has no value. If we want to solve valuable business problems then we have to design for the environment in which those problems exist, and any constraints that such businesses may impose.
Corda, a Blockchain Platform
Corda is an open platform that allows participants in a highly decentralised system to stay fully synchronized about facts that are shared between some or all of them. Superficially this sounds like a distributed database, but the key difference is the decentralisation. A distributed database is the property of one organisation or person, but a decentralised one has many owners. More importantly, unless those owners have reason to share everything with their peers then they shouldn’t have to do so.
Corda extends this. Once facts are shared into its decentralised ledger, they can be used to make other updates, potentially with other participants. Corda’s smart contracts constrain what those updates look like, so it can ensure that accidental or deliberate misrepresentation of the original facts will be caught and rejected.
Of course, there are many blockchain platforms that make claims to do this. What makes Corda more interesting from the perspective of “getting real” is that it was designed in conjunction with an incredibly challenging set of users. Corda is sometimes dismissed as a “blockchain for banks” but such observations miss that regulated financial institutions set a very high bar for success. They want to use technologies they can trust to have real business value, doesn’t needlessly share information with counterparts, that can be supported for decades, and will integrate with their existing infrastructure.
Corda has a “private by default” communication model, yet retains the flexibility to allow any set of participants to transact in arbitrarily complex ways without any impact on non-participants. It also extensively leverages the Java virtual machine (JVM) and SQL databases because of their proven longevity, broad adoption, and massive developer talent pool.
It’s easy to dismiss this as an over-conservative approach, but it’s worth asking ourselves a question. How many of the things we all take for granted actually rely on some action years, or decades in the future “just working”? If we buy an appliance with a 10-year warranty, then in 9 years we really want to be sure that any smart contract written now will still work as planned. How about buying property? We might not sell it for decades. What sort of design planning is needed in both our core blockchain platforms, and any applications written to use them, such that such longevity is both possible and plausible?
This isn’t to say that Corda is perfect, but it provides production-capable building blocks to let us solve business problems. We still have to build applications on top of it, and that’s the next challenge in getting real.
Production Applications
Let’s say we want to build a CorDapp to solve a business problem for five different organisations. We’re going to end up writing code, building tests, having user acceptance, deploying, etc. These steps are common among technology projects, with proven best practices ready for us to embrace, but decentralised systems add some new wrinkles. To understand why we have to consider what our blockchain is doing for us, but also what it’s not doing for us.
Our five organisations wish to use a blockchain because they desire a consistent view of the data they share. The logic that updates the decentralised ledger is identical for each participant and they all agree what’s being transacted and what it means. This means they’re all set on sharing the format of on-ledger representations of their transactions, and the contract code that allows for updates to take place, but they still all choose what, why, or when, they transact on a basis unique to each of them. If those decisions come from other systems then they will each need a unique strategy for integration, that matches their own established policies. Indeed, each of the five will have their own unique policies around how their systems are integrated, deployed, operated, and evolved. It’s possible that we might be able to get all five to go live on the same date, but thereafter their own policies mean it may be very hard, if not impossible, to ever get all of them to coordinate the same technical steps at the same time ever again.
We already noted that in some cases we might need to enter data into our shared ledger now, but that will still lead to correct operation years, or decades, into the future. When we start to think about what’s represented on the ledger how can we ever be sure that our understanding now will meet all the future needs? We can’t, of course. Transactions may become subject to new regulations, such as GDPR that introduced a huge swathe of restrictions, so we have to plan for evolving data in the ledger and the contract logic that controls it. While GDPR represented one particular privacy concern, when we think about sharing data with potential competitors then we’re probably also going to have to think carefully about what we share, and on what basis.
Now that we’ve anticipated the need to possibly have to update our shared data and code, we might ask who is going to decide on the needs for updates, and how will that process look? Each of our five organisations might have differing views. What happens if we decide we want to introduce a sixth or seventh participant? What if one of our original participants decides to exit this business and no longer wishes to be part of the blockchain network? What if we want to share our Corda Compatibility Zones (blockchain network) with other organisations for unrelated types of activity?
Blockchain offers many opportunities to reduce duplicated efforts across our group of participants but comes with some new challenges. Some are conceptual, some technical, some organisational, some operational, and some project governance. Left to their own devices, our group may well find solutions to all these questions and more, but the next group, and the one after that would have to re-learn the same things. What we really need is to factor out those things into something reusable, and that might in turn lead to important improvements to the core Corda platform. It’s essentially the same refactoring process that is used routinely in software development, only now some of what we need to refactor isn’t just software. There’s a clear need for engineering activity focused on spanning the needs of decentralised applications.
R3 Solutions Engineering
I thought I’d conclude with a brief introduction to the team that fulfils this within R3. R3 have quite a number of engineering teams, but one team in particular, Solutions Engineering, is set up to assist customers with building successful production systems on Corda. Unlike our exceptional Platform engineering teams, who concentrate on developing Corda, our focus is entirely on helping customers succeed with building production systems using Corda. It’s here that we get to turn individual interactions with many different teams that are building on Corda into the sort of reusable content that future developers can leverage.
Since the team was formed our work has had us think deeply about many aspects of using Corda. Here are a few highlights.
Business Networks
Business networks are the construct we use to have many different applications operate within the same Corda Compatibility Zone. We evolved the concept of business networks within Corda, providing guidance around design and governance.
As part of this we found a need for a number of business network services, so we designed and build them. To date we have abusiness network membership service to allow CorDapp participation to be managed by a specific operator, a service to check ledger synchronisation with business network peers, and a CorDapp distribution service that provides CorDapp updates to business network members. All aim to simplify the task of building and supporting decentralised applications.
Design
Design patterns are a key aspect of software development. We established patterns that can be used for many different applications, such as cash, asset exchanges, property transfers, and insurance.
One of the most exciting innovations is around the Corda Modelling Notation that helps designers to reason about Corda states, transactions and privacy. We expect the work in this area to evolve and continue to simplify the task of building robust systems that will stand the test of time.
We discussed how making blockchain real isn’t completely straightforward, but we produced a CorDapp design readiness guide to help developers evaluate their own work.
A key requirement for a decentralised application is to allow it to be showcased and tested. As part of this we have developed the Corda trials engineering framework, including building many of the capabilities in the trials deployed so far.
Production systems need to avoid introducing unwanted constraints that don’t need to exist in other approaches. As part of project Ubin, and several subsequent projects we built the first decentralised cash netting solutions that can operate safely, without placing constraints on any participants ability to do other transactions.
Deployment
A blockchain platform is of no use if it can’t be deployed on customer sites. We’ve worked extensively with customers in very challenging environments, with the Corda platform engineers, and strategic cloud hosting providers to ensure that Corda can be use in every environment we’ve encountered.
Training
If you’ve seen the Corda Developer Certification programme, then you’re looking at content from the team. In addition, we also support some of the outreach activities that have seen multiple public and private hackathons that have chosen to use Corda.
Looking Forward
We’ve looked at some of what it takes to get real with blockchain. What we’ve discovered along the way is that learning how to build and deploy successful decentralised applications isn’t easy, but right now this is one of the most interesting problems in software engineering. Like all interesting problems, things are not always straightforward. At times, supporting other teams building completely new decentralised systems can feel like being a therapist (yes, we feel your pain), but the most rewarding aspect of working with any platform is seeing what other people are able to do with it.
We’re looking forward to sharing more of what we’ve been learning, and sharing our new ideas and implementations, in the coming months. Our default is to be open, so we share as much as we possibly can, and actively welcome your thoughts and ideas. You can find more information, and ways to contact us at: http://solutions.corda.net.
Afterword
I hope I conveyed some of my own excitement about the work we do within the R3 Solutions Engineering team and why we do things this way. R3 is an incredible place to work, with an amazingly talented team, and offers the opportunity to do things that I believe will come to define the next generation of decentralised systems. If you think this is something you want to help us deliver, then you can also check out our latest job openings at https://jobs.lever.co/r3.com
