Blockchain and Distributed Ledger Protocols — a Reading List
A curated and annotated collection of insightful technical papers and articles worth reading to learn more about the origins and state of the art of blockchain and distributed ledger technologies.
Beginners to blockchain technology are often overwhelmed by the sheer volume of ideas, protocols, white papers that are out there, and it can be hard to separate the signal from the noise. I myself have been on a two-year journey, still ongoing, to identify and understand some of the important ideas in this space.
I have put together the following list based on my experiences, including running a graduate-level research reading group and teaching a course on the subject at USC. It aims for a broad coverage of different protocols rather than in-depth references on one or more of them, and as such is aimed towards a beginner in the field:
- Bitcoin: A Peer-to-Peer Electronic Cash System — the original 2008 classic by Satoshi Nakomoto that launched the revolution by combining a number of key elements together — hashed chains, public key cryptography, distributed proof of work mining with longest-chain adoption, economic incentives. (read “Bitcoin in a nutshell”).
2. Ethereum: A Next-Generation Smart Contract and Decentralized Application Platform — white paper describing the second generation state machine-based blockchain protocol launched in 2015 with a (quasi) Turing-complete virtual machine, providing the ability to do much more general computation on chain limited only by “gas” payments, using scripts referred (somewhat misleadingly) to as smart contracts.
3. Hyperledger Fabric: A Distributed Operating System for Permissioned Blockchains — a peer-reviewed paper from 2018 describing the architecture one of the most popular permissioned blockchain protocols. Unlike open protocols like Bitcoin and Ethereum, permissioned protocols are closed in that they limit participation to known, identified participants. This paper argues for the benefits of separating executing a transaction and checking for correctness from ordering consensus, and putting execution before ordering. The choice of consensus mechanism is left open for a more modular design.
4. Tendermint: The latest gossip on BFT consensus — A 2018 paper presenting a simplified Byzantine Fault Tolernant (BFT) consensus protocol. The protocol proceeds in rounds, each having a dedicated proposer. It is designed to be simpler to understand and implement, and in case of a correct proposer and timely communication it terminates in three steps. The paper provides formal proofs of the algorithm’s correctness.
5. The Swirlds Hashgraph Consensus Algorithm — Hashgraph is a DAG-based protocol from 2016, which utilizes a gossip based algorithm and provides for provable Byzantine Fault-Tolerant consensus. It is leaderless, asynchronous, and fast in the absence of faults. It provides for fair total ordering with minimal communication. Other notable early DAG-oriented protocols include IOTA, Spectre.
6. Ouroboros: A Provably Secure Proof-of-Stake Blockchain Protocol — This 2017 paper describes and formally analyzes a synchronous protocol proceeding in rounds or slots corresponding to block creation, batched into epochs, developed for the Cardano blockchain. At the beginning of each epoch, a committee of stakeholders uses secure multi-party computation to elect a random sequence of block producers for the slots within that epoch, and the committee for the next epoch. The probability of selecting a particular node as block producer is proportional to its stake. (read more)
7. Algorand: Scaling Byzantine Agreements for Cryptocurrencies — Algorand, published in 2018, provides a byzantine agreement mechanism that utilizes Verifiable Random Functions to select a subset of users in a private, non-interactive way to be involved in the consensus. The mechanism utilizes proof of stake, giving each participant a weight proportional to their monetary value, and is shown to be scalable, allowing for high transaction throughputs and avoiding computationally expensive proof of work.
8. Avalanche: A Novel Metastable Consensus Protocol — This paper written by the anonymous “Team Rocket” was introduced in 2018. It presents a novel leaderless, gossip-based, DAG-oriented protocol for probabilistic consensus that is shown to have better communication complexity and hence much higher scalability in terms of both nodes and transactions per second. Safety and liveness properties of the proposed protocol are described. Avalanche, however, assumes some Sybil control mechanism is used and does not address incentives. Perlin is one DAG-based ledger protocol that utilizes Avalanche consensus while addressing these issues.
9. Zerocash: Decentralized Anonymous Payments from Bitcoin — this 2014 paper shows how zero-knowledge succinct non-interactive arguments of knowledge (zk-SNARKs) can be used to construct a decentralized anonymous payment schme in which a payment’s origin, destination and transferred amount are all kept private.
10. Tortoise and Hares Consensus: the Meshcash Framework forIncentive-Compatible, Scalable Cryptocurrencies — This 2017 paper combines a permissionless byzantine consensus protocol based on proof of work (the tortoise) with a possibly-faulty but quick consensus protocol (the hare). It improves the average consensus time while retaining worst case security guarantees about eventual consistency and immutability. It is a block-DAG protocol.
Bonus — further reading:
11. The Bitcoin Lightning Network:Scalable Off-Chain Instant Payments — This 2016 paper describes how a network of “second-layer” micropayment channels, that defer broadcasting pairwise transactions to the main chain, can be used to enable scalable, fast bitcoin transactions. (A similar network developed for Etherum is called Raiden).
12. Atomic Swaps — A mechanism to allow trading of cryptocurrencies across different blockchain/distributed ledger platforms.
13. Cosmos: Network of Distributed Ledgers — This paper advocates the use of hubs, based on Tendermint, to allow different distributed ledgers (zones) to send transactions to each other using an inter-blockchain communication (IBC) protocol. It is an effort aimed at creating greater interoperability between many platforms and enabling easier deployment of sharding-oriented solutions for scalability.
14. Truebit: a scalable verification solution for blockchain — This paper brings a scalable computation framework to Ethereum by providing financial incentives to perform and verify computations off-chain using a novel verification game.
15. Ripple Protocol Consensus Algorithm — A BFT consensus protocol that is aimed at very low latencies by utilizing collectively trusted sub-networks. In the worst case, it can tolerate (n-1)/5 Byzantine failures, a little weaker than the typical (n-1)/3 threshold. It has been deployed widely in the financial industry.
16. Tezos — a self-amending crypto-ledger — This white paper from 2014 describes the Tezos governance-centric philosophy, beginning with a proof of stake-based seed protocol that allows stake-holders to vote for amendments including amendments to the voting process itself. A more recent 2017 paper on Tezos describes its design and implementation using OCaml, and the Michelson smart contract language with formal semantics that aims to facilitate verification and provide greater security.
17. EOS.IO — EOS.IO is a delegated proof of sake cryptocurrency and decentralized application development platform launched in 2018 which trades off the extent of decentralization and relies on off-chain governance for security, in order to provide high performance, eliminating user fees, and providing off-chain governance.
18. Bancor Protocol — this sophisticated protocol provides a way to provide automated purchase/liquidation of smart tokens at a price that is programmatically calculated to balance supply and demand.
19. Token Curated Registries 1.1 —a mechanism that aims to create and sustain token-based economy to incentivize and manage the decentralized curation of a list.
20. Augur — A decentralized oracle and token-based platform for prediction markets, with an incentive structure designed to make honest and accurate reporting the most profitable option for token holders.
Bonus+, still further reading:
- Blockstack: a layered architecture for blockchain
- Plasma: a second-layer solution for scalable smart contracts
- Dual-deposit escrow: a scheme for buying and selling digital goods without trusted mediators
- Holochain: an agent-based distributed ledger approach
- NEM: a proof of importance based permissionless blockchain protocol
- Decred: a hybrid proof of work-proof of stake protocol
- Streaming Data Payment Protocol: an application layer protocol that enables cryptocurrency micropayments and ledger-based storage
- DDM: a decentralized real-time data marketplace
- Ocean Protocol: decentralized orchestration for AI data and services
- Mimblewimble/Grin: a proposal to increase efficiency and privacy of transactions and its implementation as a new blockchain protocol
- Bulletproofs: improving the performance of zero-knowledge proofs, useful for confidential transactions. (more.)
- R3 Corda: a distributed ledger focused on financial transactions
- Dfinity: a protocol aiming to build a decentralized virtual blockchain computer
Have I missed something? Are there better links to provide? Please let me know what you think.