Blockchain
Learn why blockchain technology has the potential to transform every industry
Emerging blockchain solutions point to a future of dependable decentralized trustworthy transactions that’s within reach
One of the lessons I’ve learned from writing about blockchain is that you must provide a bit of an introduction on the technology — otherwise, I’ll lose 80% of the audience. This is understandable given the complexity of blockchain technology and the confusion around different platforms, networks, tokens, and applications.
Much of the public’s attention is focused on the rise and fall of cryptocurrency values and the huge amount of dollars flooding into the space. This is a shame — because under the hood of blockchain is some impressive architecture and vision that promises to create a huge wave of innovation and, along with it, a fair bit of disruption.
Before I get there, and in the interest of the first rule of blockchain, here are some key points about cryptocurrency and blockchain to serve as a base.
Blockchain 101
- Cryptocurrencies are separate from blockchain technology. The first is a digital representation of perceived value; the second is a method for distributed transaction processing and storage.
- At the core of a blockchain platform lies a distributed transaction processing engine that validates and cryptographically seals transactions.
- Blockchains consist of sets of transactions that are sealed within a “block” using cryptographic algorithms. Successive blocks are linked to the preceding blocks by using hash values such that there is a sequential chain that is largely permanent and immutable.
- Blockchain networks are comprised of near identical nodes operating in a distributed but independent manner. The network of nodes is used to validate the transactions and encapsulate them within blocks in a chain.
- Blockchain transactions are maintained in a distributed ledger that is kept in conformity across the nodes in the network via consensus algorithms.
- Blockchain miners are operators of nodes in the network and are rewarded for forging the blocks via cryptocurrency. The first miner who arrives at a hash that is within a certain parameter receives the reward for that block.
- If additional transactions are allowed within the blockchain platform, minors also receive rewards for processing transactions and storing accompanying data.
Blockchain Use Cases and Potential
Okay, now that we have the basics covered and we’ve established that blockchain technology allows the creation of distributed transaction processing and storage networks — it should become easier to see the potential.
Having a public or private network that allows for independent verification of records of transactions fills many needs across a number of industries — records that have been hashed and connected to create an immutable chain. This is especially true if one highlights the premise that the centralized processing and storage dominant today requires trust in each central organization — trust that many people now and certainly in the future are not likely to share.
A list of use cases is a post in itself and so I won’t catalog them here but if you are interested, there are many articles that touch on the subject –both in theory and in practices. One of my recent favorites is provided by Jeremiah Owyang, Founder, Crowd Companies in graphic form.
The Case for Blockchain Technology
If you need more convincing, a simple look at the Bernie Madoff fraud shows how blockchain technology might have stopped it in its tracks. In 2008, Bernie Madoff confessed to operating a $64.8B ponzi scheme– yes, billion with a ‘B’. It is considered to be one of the largest financial frauds in history.
In the subsequent investigation, it was found that his asset management firm had been making up trades and posting false returns for years, if not decades. The shock and fallout among clients was devastating and the fear among general investors was palpable and, to some extent, even lasts to this day.
Regulators missteps, or malfeasance, played a primary role in allowing the fraud to continue. But in an alternative universe, if all trades had been recorded within an immutable and accessible blockchain record, then it would have been difficult, if not impossible, to pull off.
Verifying trades against the blockchain of transactions would have made it readily apparent to regulators if in fact the trades had been made. It would also allow account holders to reconcile their accounts against the transaction log.
If the trades were in the blockchain and if they were the last owners on record of the securities then their account statements could be believed. Counterparties to trades could also be verified in similar manner.
Instead, regulators were misdirected in what they were looking for and were never able to see the massive scheme even when it was in front of them.
[Madoff] stated that Ostrow and Lamore “never really got into books and records as related to stock records or DTC records.” Madoff stated that “they never even looked at my stock records” or did a “box count.”
He said he was “astonished” that they didn’t ask for DTC records, and stated that only a regulator could get those records from DTC, and the SEC would “have to go to DTC.” He added that DTC does not have separate accounts for each customer, but rather, provides a global report, but stated that if they went to DTC, they would’ve seen his market-making position, and that it “would’ve been easy for them to see” the Ponzi scheme.
Madoff stated that the SEC could’ve gone to counterparties, and if they had, they “would’ve seen it,” adding, “they didn’t do any of that.” He stated that “it’s the only thing to do,” and clarified, “If you’re looking at a Ponzi scheme, it’s the first thing you do.”
– Henry Blodget, Business Insider, Oct. 31, 2009
“Madoff On ‘Idiot’ SEC Investigator: He Thought He Was ‘Lieutenant Columbo’”
The Blockchain Stack
If you accept the premise that blockchain technology will be a large factor in the future of transaction processing, then it makes sense to look at the vision that blockchain pioneers have painted. We’ll do this by looking at a some early architecture diagrams as well as diving into some of the emerging components and networks.
One of the first places to look (aside from the initial Bitcoin whitepaper) is the vision painted by Vitalik Buterin, the creator of the Ethereum protocol. Below is a diagram that contains an early model for a cryptocurrency ecosystem. The layers demonstrate an expansive vision for what is needed to handle complex transactions and interactions. It lists a significant amount of components that would all need to operate within a decentralized network.
As you can see, it is a complex vision but has a long way to go before it can be realized — both in action by developers as well as in acceptance by the marketplace.
Blockchain Core–Protocol, Interfaces, Networking
A more constrained view of the base layers is shown below. It was produced by members of the core Ethereum team shortly after the Ethereum launch. The base layers contain the protocol as well as the interfaces, networking, and test harnesses.
This core view provides the fundamental principles and interactions that control the transactional processing logic as well as the forging of the blocks. A concept called smart contracts is at the heart of the flexible processing capability within Ethereum — see my early post for more on this subject.
A key point when viewing blockchain architect diagrams and looking at various components is to have the word “trustless” in mind. While an architecture or stack diagram around transaction processing is not all that ground-breaking, much of what is being done within public blockchain platforms is unique.
The reason is that the blockchain systems are designed to be trustless — which means designed to perform an action and operation in a manner such that trust between parties is not a condition of the action or operation.
In other words, when trading for Bitcoin or Ether, it is not a requirement that one trading partner ascertain that the other trading partner has sufficient currency to trade. Rather the blockchain itself provides the ability to have ownership verified prior to the commitment of a trade.
Whether a transaction has been performed at a prior date is not up for determination by a single central authority — but instead determined by using the distributed ledger as the source of truth.
This trustless nature should be a lens that one uses when viewing other layers and components within blockchain platforms and networks. Decentralized is also a descriptor to have in mind but “trustless” is a key differentiator from almost all other platforms and networks and which is why blockchain has gained such strong adherents.
Blockchain Components
The base layers of a blockchain platform addresses transaction set-up and processing as well as forging/creation of each block in the chain. This base or kernel is sufficient for simple asset transactions — one party trading an asset to another party.
More complicated transactions, however, require deeper thinking as to how to construct various workflows (see the chess game explanation in my prior post). Attention is also needed to address large amounts of data that might be involved with the transaction.
To handle this added complexity and satisfy more ambitious use cases, additional components and networks have been proposed. Below is a rundown of a number of the more important ones.
Note the use the terms ‘layers’, ‘components’, ‘applications’, ‘platforms’, and ‘networks’. The reason is to differentiate the conceptual aspect of a technology (layer or component), from a technical implementation (application or platform), from a set of independent nodes running an application or platform (network).
Identity Solutions
Secure and verifiable digital identities are critical in the operation of a robust blockchain network. Identities are needed for parties making transactions as well as the assets forming the basis of the transactions.
There are a number of issues regarding identities of transaction parties. One is whether the parties are identifiable or anonymous. Most blockchain platforms will allow for anonymity although the network structure or regulatory requirements on parties connected to a network may impose off-network association of real-world identities with on-chain IDs.
The IRS, for example, has put pressure on Coinbase and other online repositories for data that would tie users to their Bitcoin, Ethereum, or LiteCoin account IDs. Another complication can arise in entities may want or require multiple identities and so any structure would need to accommodate this.
For example, one ID might be tie to a identifiable name/person (a blockchain ID that ties to a drivers license, voter registration, or a passport), others might be semi-private — known only within a particular closed network, and still others might be kept anonymous, created to establish a legitimate entity but one that is not traceable to the off-chain entity.
As for identities for assets, the issues are equally as complicated. It is one thing to transact with a system’s cryptocurrency as the system is able to authenticate legitimacy of that currency. It is different, though, for non-digital assets as well as digital assets that are not hard-wired into the transaction processing engine.
With regards to non-digital assets, establishing a digital ID that matches a physical asset can be problematic. A painting, for example, would require one or more photographs plus a detailed description (title, artist, media type, size, distinguishing marks) and/or condition report and yet there might still be questions as to legitimacy.
An automobile with a VIN number might look as if it is easy to identity. The problem is that a single VIN number just scratches the surface. Cars are made up of a number of parts and subcomponents, many of them also uniquely identified. A single automobile might contain up to 30,000 different items (if one takes it down to the individual nut and screws). If a car has repairs made to it and parts are replaced, the manifest would need to be modified to reflect these changes.
And so as you can see, blockchain identity is complex. It is also in it early stages but many parties are looking at it and working to address the complexities. Microsoft, for example, is actively involved in blockchain identity and putting outsome pioneering thinking.
As progress is made and standard approaches adopted, blockchain networks will become far easier to use. When ease of use increases, the number of transactions committed to blockchains will also increase and blockchain technology will seem less foreign and strange and more matter of fact.
If you doubt this, then think about the impact digital cameras within phones had on use of photos on the web. Digital cameras with ubiquitous network connections made it far easier to upload and share photos. This allowed for the emergence and exponential growth of Facebook, Instagram, Pinterest, Snapchat, and other social media platforms. The same thing will happen with blockchain applications once entity and asset IDs are in forms readily usable within blockchain networks.
Off-Chain Data Validation
Off-chain data validation is another area that is key to addressing more complicated blockchain solutions. The amount of data that can be stored on-chain is limited. There are practical governors imposed by block size limits as well as the fees needed to create a blockchain transaction.
As with on-chain transaction processing, on-chain data storage comes with a cost. Charges must be paid by the transacting party at the time of a transaction to cover the cost of storage. This StackExchange post provides more information on blockchain data storage — although note that the charges detailed in the explanation might be outdated given recent price increases in Ether.
It is very much an architectural decision as to what information to include within a blockchain transaction — obviously the entities involved in the transaction and the assets traded but other data and aspects of the transaction might also be recorded in the on-chain data.
Data elements that are critical to a transaction but are large in size, however, are not going to be able to be included on-chain–at least in whole. Examples might include a legal agreement or contract, accompanying documentation for a large asset purchase (a house, for example), a lengthy manifest for a container shipment, or photos and a condition report of an artwork.
Not being able to include them outright as part of the on-chain transaction, however, does not mean identifying elements of the data cannot be anchored or pinned into the blockchain as part of the transaction. In other words, the full documents may not be included but it is possible to include a representation that is unique and can be verified at a later date.
There is a protocol and evolving data proof network developed by a company called Tierion that provides a turn-key way to anchor data to public blockchains, The network essentially takes a hash of any form of data — a document, set of files, database records, .csv, .xls, or other file format — hashes in a timestamp, and then pins or anchors the resulting hash along with other hashes within a transaction on Bitcoin and Ethereum.
In this manner, they are able to provide a record of a transaction as well as generate a proof of data. At any point later, it is possible to verify if any data at hand matches the original data. What this means is that if you have a contract or set of financials in hand, you can verify if that contract or the financials are in their original forms by comparing them against the hash in the blockchain transaction. While this network does not provide storage of this ancillary data — this is left as an exercise for the transacting parties — it does provide a way to determine if any stored data matches the data that was originally part of a prior transaction.
Full disclosure: I am close with the folks at Tierion and am a supporter of the company and the technology. It provides a ready use case for blockchain technology but can be used for any solution requiring a proof of data independent of any blockchain connection.
Off-Chain Data Storage
In digging into the first diagram above, you’ll note that off-chain data storage is included within the layers of the ecosystem. For example, Swarm appears prominently in one of higher level layers. It, along with IPFS, are two protocols/ solutions that are looking to address this need.
Swarm
Swarm gets a far amount of attention in blockchains circles as its origins are more native to blockchain platforms than IPFS, although IPFS has been around a lot longer. A set of orange papers was released in May 2016 that describes the technology behind Swarm.
Here’s a description of Swarm that appears on the Swarm github page.
Swarm is a distributed storage platform and content distribution service, a native base layer service of the ethereum web 3 stack. The primary objective of Swarm is to provide a sufficiently decentralized and redundant store of Ethereum’s public record, in particular to store and distribute Đapp code and data as well as block chain data. From an economic point of view, it allows participants to efficiently pool their storage and bandwidth resources in order to provide the aforementioned services to all participants.
From the end user’s perspective, Swarm is not that different from WWW, except that uploads are not to a specific server. The objective is to peer-to-peer storage and serving solution that is DDOS-resistant, zero-downtime, fault-tolerant and censorship-resistant as well as self-sustaining due to a built-in incentive system which uses peer to peer accounting and allows trading resources for payment. Swarm is designed to deeply integrate with the devp2p multiprotocol network layer of Ethereum as well as with the Ethereum blockchain for domain name resolution, service payments and content availability insurance.
IPFS
IPFS stands for the InterPlanetary File System and is a peer-to-peer file storage distribution protocol that uses addressing conventions based on the content so that it can be server independent. This capability means that addresses can be permanent and flexible as opposed to tied to a particular server or location. It’s history predates blockchain technology but it’s implementation some of the same requirements that blockchain developers propose.
Here’s how Techcrunch described IPFS a few years ago, “IPFS also addresses security problems that plague our HTTP-based Internet: content-addressing and content-signing protect IPFS-based sites, making DDoS attacks impossible. And to help mitigate the damage of discontinued websites, IPFS also archives important public-record content, and can easily store important, public-record content.” IPFS is mentioned in blockchain forums although I am not aware of any production uses at this point in time.
For more information on the Swarm and IPFS, here is a StackExchange post that compares the two.
Given the challenges with distributed storage, it is certainly early in its evolution. It not only unclear as to what the economics will be for distributed storage but also how the market will take it. Processing transactions is one thing, especially if identities are anonymous, storing data is another story — even it that data is encrypted, partitioned, obfuscated, and in other ways made secure and anonymous.
One possibility is there could be a melding of private storage solutions with public distributed networks. In other words, Dropbox, Box, or OneDrive could create their own tokens and use these to power Swarm-like capabilities using edge storage capacity among their existing users (i.e. using their existing users local storage as the network.)
Alternatively, a startup (not unlike Pied Piper from the Silicon Valley TV show) could arise by making use of mobile phones, smart refrigerators, and other devices as an edge-based storage network. #kidding #notkidding
Payment Channels
Recording transactions using blockchain networks has costs both in terms of speed and throughput — as well as in the physical cost to record each transaction.
Blockchain networks are well suited for recording large one-time transactions such as the transfer of a house, a car, a piece of equipment, or other notable purchase. They are not, however, great for small transactions or in cases where transactions take place on a more frequent basis.
An approach called a payment channel has been proposed to address these types of situations and a few networks are in various stages of development within a few blockchain platforms.
An example where a payment channel might fit is in the case of a pay-per-view capability. The way it works is that an initial deposit is created on-chain transaction but with a time lag. With each view or download, a signed transaction gets created but it is maintained in an off-chain network. Each new view/download creates a new transaction that replaces the prior transaction but with increasing amounts to be transferred.
It is not unlike a gift card where with each use another amount is deducted from the balance although in this case, the balance doesn’t get hit until an event forces the last transaction to be processed on-chain. Either party can trigger on-chain recognition of any duly signed off-chain transaction at any point in time.
Another use case is a loyalty card, where with each use of a token, a reward is provided in the form of a portion of a token but the successive rewards are kept off-chain until flushed on-chain.
“[Payment] channels are an important technology that has the potential to greatly improve the scalability and privacy of many categories of blockchain applications; in conjunction with sharding and other privacy-preserving cryptographic technologies, they are an important ingredient in helping decentralized systems to achieve the properties that mainstream individual and institutional users expect and deserve.”
–Vitalik Buterin, Ethereum Creator
The Lightning Network
The Lightning Network is a payment channel network for the Bitcoin blockchain and intended to provide transactional capabilities at high volumes and high speed. It works by recording an initial transaction within the Bitcoin blockchain network — a payment between one party and another, for example. Subsequent amendments by the trading parties, however, are made off-chain in the Lightning Network. The amendments are stored in the form of blockchain-parsable smart-contract scripts with only the most recent off-chain amendment active within the network.
Time-locks are used on the transactions to allow for a delay in the recognition of the initial deposit as well as the recognition of the interim adjustments. This lag in recognition allows for the most current entries within the off-chain network to be closed out at any time, thus cancelling out the deposit and only recognizing the true amount of the transaction. In the case of non-cooperation, the blockchain is the arbiter using the initial state and the most recent agreed upon transaction.
The Raiden Network
The Raiden Network is a payment channel network for Ethereum. Like the Lightening Network above, the Raiden Network establishes an off-chain state network to provide a set of properties that adds to on-chain Ethereum capabilities. In this model, privately signed messages within the Raiden network provide for a transfer of value. The messages are maintained via the peer-to-peer network in combination with an initial on-chain deposit. The messages would in a similar manner as described above using time-locks and the most recent off-chain transaction to establish the current balance (once the transaction is committed on-chain).
The creators list a number of uses including micropayments for content distribution, ads, and subscriptions; on-demand marketplaces for bandwidth, storage, cpu processing, energy, or sensor data; and token systems and private currencies.
Communication Networks
Most current communications protocols and networks used in business allow for the identification of communicating parties and the tracing of locations. Various blockchain platforms, however, have protocols/networks in the working or proposal stage that are designed to obscure locations from parties communicating within the network, thereby shielding identifying elements of the parties. The idea is that while blockchain ID can be created so as to anonymous, location information included as part of a communication message could end up providing sufficient information so as to unmask this ID.
Note that the protection of identities does not necessarily imply illegal trading or wrongdoing. Complete or partial anonymity is a key and common element in most trading situations, especially during the setup and offer phase.
For example, auctions exist largely on anonymity. Same with trading stocks or options. At each offer or trade, identities are not broadcast to the entire network. Instead, auction houses and clearing houses act as central authorities protecting the identities of the parties but allowing for trustworthy transactions.
With blockchain networks, there is no central authority that can buffer communication information. As a result, any location information conveyed as part of a broadcast communication to the network could compromise the identity of the blockchain participant. Therefore, there is a need for a communication facility that can obscure location information and preserve anonymity.
Whisper
Whisper is a communication protocol for Ethereum that allows clients to pass messages that do not need to be permanently stored in the blockchain. Per the documentation, the Whisper protocol is intended to maintain the security and efficiency in sending, multicasting, and broadcasting messages.
Outcome Authentication
Yet another challenge with decentralized transactions is how to authenticate an outcome of an event that is off-chain in a way that it can be used to determine the outcome of an on-chain transaction.
The outcome of an online activity — a chess match, a poker game, or other activity — should theoretically be arbitrable within a blockchain network by assessing the current state of the game and along with the prior moves or actions. Establishing the outcome of an off-chain activity is far more difficult.
In the case of a bet on a football game, it could be recognized and paid out provided both parties agree as to the outcome. If one party, however, does not agree on the outcome — whether it be out of in-attention, spite, bad faith, or otherwise, then a mechanism is needed to provide the outcome that will close out the wager.
The 1970s movie The Sting shows why it is a necessity to have a way to obtain outcomes in an authenticated way. The two main characters set up sham betting hall to con a mob boss. They use a scam called the wire. This scam originally started with delaying news thereby allowing bettors to front-run the outcome. It later evolved to a more elaborate ruse that used fake betting parlors and illusion of advanced information to con gamblers.
The wire was most popular in the early 20th century, when horse and dog race results were sent to betting parlors via the telegraph. It began in the early days of telegraphy, when it was possible for a telegraph operator to delay racing results long enough for a few confederates to make bets, and make good money.
As telegraphy became more widespread and local bunco squads put protections against delaying results into place, grifters began to create the illusion they could delay results, giving rise to the wire con. The con is time and personnel intensive, requiring a team of cons to set up and operate a fully staffed, but fake, betting parlor. Critical to the con is the “roper”, a grifter who must lure the mark in with promises of advance knowledge and guaranteed wins.
– The Wire Scam, Leverage Wiki
Creating a system that can provide outcomes in a reputable way for use with a decentralized and trustless transaction processing system is not a simple matter. Two of the more prominent networks working on a solution are Augur and Gnosis.
Augur
Augur is also prediction market operating as a decentralized network on Ethereum. They provide a facility to trade virtual shares on the outcome of real-world events. The concept of a prediction market is not new but the theory is that the decentralization provided by a blockchain network reduces counterparty risk as well as the threat that might be imposed against any single central authority. For an more in-depth view, take a look at their FAQ as it contains a solid run-down on how it works as well as the issues involved with in operating a decentralized prediction market.
Gnosis
Gnosis is also prediction market operating as a decentralized network on Ethereum. Essentially they are creating a platform whereby there are providing mechanisms for aggregation information about the expected outcome of a future event as well as allowing for transactional activities based on that outcome. Uses include financialization, insurance and hedging, price discovery, performance incentivization, and sports betting to list a few.
The Gnosis architecture contains a number of interesting elements, one of which it termed an “oracle.” An oracle is a facility that determines the outcome of wagers. They list four categories of oracles:
- On-Chain Oracles
- Centralized Oracles
- Decentralized Oracles
- Hybrid Oracles.
The discussion in their whitepaper provides a fair bit of detail on the nature and circumstances for each type. The paper is worth reading to get an idea as to how a simple workflow in a centralized workflow — a bet at a sports book in Las Vegas — becomes far more difficult to administer in a marketplace designed to be decentralized, permissionless, and trustless.
For comparisons between Augur and Gnosis, see posts here and here.
Blockchain Platforms — Now and Future
The current focus in the media on the cryptocurrency element of blockchain has taken away a fair amount of attention away from the underlying technology. This is especially true as it relates to the more complete vision laid out by blockchain pioneers.
The growth of the current networks is pretty amazing. Think about it. A few years ago, not many would have given odds that large decentralized networks consisting of thousands of independent nodes transacting millions of dollars in trades would be operating by this point in time.
The economic incentives provided by cryptocurrencies to support transaction processing is a case study that will mark this point in time in the history — much like other economic IT breakthroughs have done so in the past.
The blockchain story will fit in easily alongside the story of the early PC software manufacturers who sold software on floppy disks and created billion dollar markets.
It will find its place in company of open source pioneers who created massively popular non-commercial software along with huge companies built on top of this foundation.
And it will slot in with cloud computing and serverless companies who derived ways to not only make utility computing a reality but reshape IT architectures and deployments for many years to come.
Blockchain technology has the same potential to transform almost every industry. The ability to create dependable workflows and allow for trustworthy transactions in a manner that is decentralized and trustless is a huge vision — but one that is not unattainable.
The progress in blockchain technology is real and the vision for the future is certainly within reach.
FWIW, I’m helping to produce a Blockchain Workshop on Friday, August 4, 2017 in San Mateo, CA. Details can be found here.
This post is a follow-on to a recent blog on the similarities and differences between blockchain technologies and serverless processing.
Thanks for reading! If you like what you read, hold the clap button below so that others may find this. You can follow me on Twitter.
