Blockchains Are Broken. Here’s why.
This article contains the first two parts of my four-part mega-article, Defusing Crypto’s Ticking Time Bomb. For convenience, I have also turned part three and part four into their own separate articles.
Part I covers the basic elements of blockchain infrastructure — block producers and the peer to peer network. Part II, which makes up the bulk of this article, covers the major problems in the blockchain space in a first-principles manner. The remaining two sections which are not present in this article cover the current solutions to these problems (Part III) — and a new set of solutions which come at things from a very different angle (Part IV). The content here is necessary background to those sections.
I hope you enjoy it, learn something, and push past any confusion you might encounter! If you could find the little clapping icon👏 on the bottom of your screen and hold down your click button on it — you will be able to give this article up to 50 claps. I would appreciate you doing so, as this took a lot of time and effort to create.
Part I — The Players
Blockchains have an infrastructure problem. This includes your favourite chain. Before we can get into what exactly it is, we need to know what the key elements of a blockchain are. We can outline them by looking at Bitcoin, which is a good template for the space.
Bitcoin has two key infrastructural roles, which each play a distinct function. I will outline them here. Feel free to skip Part I if you are already comfortable with basic blockchain network design.
Block Producers (miners, stakers)
Block producers are responsible for guaranteeing the security of the network. In Bitcoin, it is through hashing (spending electricity) and block producers are called miners. In proof of stake, it is through risking capital (stake) and block producers are called stakers. I’m going to stick with Bitcoin terminology for simplicity — but remember that the general process outlined here applies to both Proof of Work (PoW) and Proof of Stake (PoS) networks.
First, users want to interact with the network — and so broadcast their transactions out to nodes. Miners (who run ‘mining nodes’) receive some user transactions directly from their own node, but they also receive transactions from the peer to peer network of nodes. Miners verify transactions when they receive them, rejecting ones which violate the rules of consensus. Inbound transactions pile up in the block producer’s mempool (or, transaction pool) over time.
Miners then compete amongst each other to put these transactions on the blockchain (to confirm the pending transactions). They bundle a handful of transactions from their mempool into a ‘block’, which will get put on the chain if they successfully win a proof of work competition. The miner who solves the proof of work puzzle broadcasts their winning block around the network, and the other miners add the winning block to the chain (synchronise with the rest of the network), before they start mining on the next block. Whichever miner produced the winning block (added transactions to the blockchain) gets paid.
Mining (or staking) exists, above all, to make producing a block costly. If producing a block was not a costly affair, the network would be susceptible to a range of attacks and failures (sybil, spam, double spend, constant forking — and more) and would fail to be a viable blockchain. Mining is the security mechanism of Bitcoin.
Note: Multiple miners can have the same transaction in their mempool or proposed block — but when a new block gets added to the chain, the Bitcoin software ensures mempools and other proposed blocks are updated so that duplicate transactions are removed and replaced with new transactions.
The Peer to Peer Network
The peer to peer (p2p) node network, broadly categorised, is comprised of constantly running ‘archive nodes’ and ‘full nodes’ — both of which perform critical network functions. While mining nodes can be either archive or full nodes, the p2p network itself consists predominantly of non-mining, full nodes.
Different blockchain communities give all the varying types of nodes different names, and this can make things quite confusing. Bitcoin’s definitions are different to Ethereum’s definitions, but how we define ‘node’ doesn’t really matter as long as we know what exactly we mean by the word. So to simplify things as much as possible, in this article I will employ the following terminology:
- If you mine or stake, you run a mining/staking node
- If you collect, verify and propagate all new transactions and blocks, you are a full node. I might call you a ‘routing node’.
- If in addition to the above, you store (and keep up to date) the entire history of the blockchain — from the first block to the latest block — then you are an archive node. Under this definition, all archive nodes are full nodes.
- If you don’t do any of these things, then you are a ‘light node’, and are not a crucial part of the peer to peer network. For example, the average Bitcoin wallet is a light node, which only connects with the Bitcoin network to broadcast its own transactions to it. It does not contribute to the network infrastructure.
A ‘node’ could perform any combination of the first three tasks just listed. But when I talk about ‘nodes’ on the p2p network, I am at a minimum talking about nodes which perform the second function above of collecting, verifying and propagating new transactions and blocks. For the more technically inclined, when I say ‘verify’ I do not mean validate headers. I mean actually verifying blocks and the transactions therein.
It is worth mentioning that although full nodes only store a portion of recent blockchain data, they amount of data they store varies depending on their hardware constraints. Some could hold up to two years of data, while others might only be able to hold up to six months of recent data. If access to data not held by full nodes is needed for whatever reason (and there are many), then archive nodes are relied on to serve that data.
Now, there are two things that the nodes on the p2p network do.
Role 1: Routing Work
The basic (and highly important) task which nodes perform is collecting transactions and new blocks which have been broadcast to them — verifying them — and then sharing them with both other nodes and block producers on the network.
Here’s how it works:
You send somebody some Bitcoin from your wallet. What this does, is broadcast a Bitcoin transaction out to the p2p network of nodes, which are constantly running. Your transaction gets picked up by a node, which then checks that it is valid — meaning, that it doesn’t break any of the consensus rules such as “you can’t spend any Bitcoin you don’t have.” It then shares it with as many other nodes as it is connected to. All the nodes on the peer to peer network which have your transaction do the same — verify and broadcast it amongst more nodes in the p2p network. This gossiping of your transaction around the p2p network ensures that as many nodes receive the transaction as possible. In addition to broadcasting the transactions they receive to one another, nodes also broadcast them to as many Miners as they possibly can. The more nodes a transaction reaches, the higher the confirmation that it is valid (because of independent verification), and the more miners it will reach. This collection, verification and propagation of transactions is the basic function of the peer to peer network.
The peer to peer network is the backbone of a blockchain. It is where a blockchain’s decentralisation comes from. If the majority of nodes on the p2p network are honest, they will independently verify transactions and new blocks — and ensure that consensus rules are enforced. Also, by acting as an information sharing network which propagates new data and blocks to the entire network, it helps create an open network which allows anyone to participate in consensus.
There is a very simple reason why Satoshi designed the peer to peer network to gossip your transaction around to all other nodes. The reason is: if your transaction reaches multiple block producers, then its chance of making it onto the blockchain increases massively. If your transaction only reached one miner (or only a subset of colluding miners), then it would be possible for that miner (or group of miners) to leave your transaction in its mempool(s) and never put it onto the chain. If the miner(s) decided, your transaction would never go through, and you will have effectively been censored. Which would be a failure of the blockchain, as it would lose the crucial properties of openness (that anyone can use the blockchain) and censorship resistance (that nobody can interfere with users’ transactions). But if your transaction was broadcast to multiple miners who are competing to put it on the chain for money (they collect a transaction fee), then your transaction will eventually be added to the blockchain.
“The purpose of the bitcoin network is to propagate transactions and blocks to all participants” — Andreas M. Antonopolous, ‘Mastering Bitcoin’
There is another (related)reason for the p2p network. And it’s that block producers, who run their own nodes (archive or full), are not actually incentivised to share the transactions they collect with one another. Even when they are willing to put the transaction they collect on the chain. This is because, to block producers, each transaction represents real value in the form of a fee which they collect if they put it in onto the chain. Would you share transactions around if they represented profit to you? You’d be better off just receiving them from other nodes and miners who are kind (or stupid) enough to send them to you, and then never share any yourself. Then, you don’t have to incur node-running costs, but still get all the benefit from free riding off the work done by the p2p network. Since the goal of miners is to make money, this logic is perfectly rational. And again, this is where the p2p network comes in to ensure that transactions are shared across the network to multiple miners.
In short: Miners and Stakers collect transactions from the network of nodes and compete to put them on chain. It’s important that your transaction goes to as many block producers as possible, because otherwise a block producer could decide to never put it on chain, and you could be censored. Block producers don’t have an incentive to share transactions amongst one another either, because for them transactions represent profit.
The more nodes a blockchain has, the more decentralised it is. Greater decentralisation (assuming the nodes are good actors) ensures that consensus rules are not violated (that the blockchain is not broken). Furthermore, the greater (good-actor) decentralisation is, the higher the guarantee that your data will make its way onto the chain without censorship. If your transaction only reached one node, the number of miners which your transaction will reach would be much smaller than if all network nodes broadcast it out to the miners they are connected to. This is why nodes propagating transactions (data) across the p2p network is crucial for network functioning.
However, all of these benefits of ‘decentralisation’ are only benefits if we assume the nodes are good actors. More decentralisation doesn’t necessarily mean that we get the properties we want. Decentralisation is merely a form of network topology. What we’re really after is p2p nodes which help to preserve network openness (through transaction + block routing) and security (through transaction verification).
Now, let’s move on to the second network function which nodes perform.
Role 2: Data Storage
There is a second, crucial function which nodes in the peer to peer network perform — and that is storing historical blockchain data.
Nodes which store the full history of the blockchain are called ‘archive’ or ‘archival’ nodes. Archive nodes store blockchain data from the first block all the way up to the most recent block, and continually update the chain with every new block produced. They also send historical data to other nodes who need it. Most nodes on a given network are not archive nodes, as storing and continually updating an entire blockchain’s worth of data requires you to purchase sufficient hardware for data storage and then constantly maintain it — which is a cost most people prefer not to incur. It is also worth mentioning that, while significantly cheaper than archive nodes, full nodes themselves are not costless to run. Full node runners incur the direct and opportunity cost of the hardware, the cost of the portion of chain data which they do store, and the cumulative cost of the electricity it takes to run the node indefinitely.
A key piece of knowledge here is that in order to verify a transaction, nodes need to have a good amount (but not necessarily all) of the transaction history on the network. If a node receives a transaction which says “Pay Satoshi 1 Bitcoin”, it will need to have enough of the transaction history of the blockchain to figure out if the person who broadcast them the transaction actually has one bitcoin to spend.
Full nodes only need to know the current balances of everyone on the network to be able to verify new transactions. But to know the current balances, you need historical data. Once a full node is synced up, a lot of historical data can be pruned because only the current state is needed to verify transactions. I say most, because some amount of recent data is needed to resolve temporary forks which occur as the chain runs — otherwise a full node would have to rely on archive nodes to keep up with the chain (Ethereum-oriented explanation here and here).
As mentioned before, the amount of data which a full node (a node which verifies new transactions and blocks) stores depends on its hardware constraints. But it is generally a non-trivial amount, and this amount will be larger for blockchains which are more scalable. Since nodes need to verify all the transactions within newly produced blocks to ensure that they are valid and can go onto the chain, blockchains which process more data (which are more scalable) will generally have greater storage requirements for full nodes. In other words, Chains with higher throughput will require node-runners, full or archive, to hold on to more data so that they can properly verify transactions.
Because archival nodes hold all the historical states of the blockchain (not just the current state), they can be used in application development on smart contract chains. Any decentralised application (dapp) which would require querying past states of the blockchain will rely on an archive node. Furthermore, non-archival nodes often rely on them for historical data. In a sense, archive nodes are the blockchain. Without them, there would be no full copy of the ledger.
In an ideal world, your favourite chain would have lots of archive nodes (lots of full copies of the blockchain) spread out all around the world, all run by different parties — making the blockchain and all of its (your) data well-protected. But ultimately, how important archive nodes are to a network depends entirely on how much the network and the blockchain’s participants need historical data for their varying purposes. To the extent there is utility in having access to historical data, archive nodes provide utility to the network.
Interestingly, Ethereum will be pursuing a vision where by trying to make running a full node as simple and cheap as possible (and introducing data sharding), the blockchain could (hypothetically) be stored without archive nodes and instead reside on lots of non-archival nodes run by regular network users, each doing a bit of the work — but with no single node being relied on to store a given piece of data. We’ll discuss and evaluate this approach later on in another article.
To recap: full nodes and archive nodes store blockchain history. There are nontrivial costs that both types of node operators pay for. Only archive nodes store the entire blockchain history in perpetuity — and this is a significantly more costly task. Every transaction, account balance or smart contract that has ever been created is on your favourite blockchain exists in (hopefully!) multiple copies across a distributed network of archive nodes who keep it constantly updated with every new block, and store every piece of data ever put onto the chain.
Part II — The Problems
Now that we’ve covered the basics of what blockchain infrastructure is, it’s time to start discussing the problems surrounding it.
The Volunteer / Infrastructure Problem
Most of the problems which follow here derive from just one fundamental problem — which is that Proof of Work and Proof of Stake blockchains pay for mining and staking, but nothing else. The entire p2p network, comprised of archive and full nodes, is provided essentially in its entirety by volunteers. Although it can be stated simply, this fact has some very serious implications for the blockchain space.
The dominant consensus mechanisms we have come up with — Proof of Work and Proof of Stake — pay block producers for producing a valid block that is appended to the blockchain. Block producers are paid with some combination of A) transaction fees inside of the block, and B) newly minted coins. But no other form of work other than mining and staking is paid for. I’ll also mention here (though I’ll cover it later) that most cryptocurrencies which claim not to be ‘inflationary’, including Bitcoin, have not yet reached their supply caps, and thus rely on inflation and fees to fund network security.
Again: our chains only pay for mining and staking, not any of the other network functions which are needed to keep the chain alive and functioning the way we want. Those are provided for by volunteers.
- The critical work of collecting, verifying and sharing transactions & blocks (essential for security and network openness) is not paid for
- The critical work of data storage and making historical blocks available is not paid for
- The p2p network providing a bulwark against miner centralisation of power (fork resistance through enforcing consensus) is not paid for.
- The provision of user-facing network services and access points is not paid for
- Other security benefits such as helping to resist selfish mining attacks by providing a shared view of consensus data are not paid for
Upon first hearing this, an instinctive reaction is: “well why don’t we just pay nodes, as well as miners/stakers?” Several problems immediately emerge. How do you price the work of data storage which nodes perform? When an archive node stores data, it stores it forever and so incurs a perpetual cost. How much does data which needs to be stored forever cost? Maybe it needs to pay rent? How can we price the rent without a central planner? Full nodes also store relatively large chunks of data indefinitely, and are not compensated for this. Furthermore, how much is a fair amount to pay nodes for collecting, verifying and propagating transactions? Nodes also do that kind of work, too. All of these are very challenging questions — and only the tip of the iceberg.
In addition to these highly challenging problems, an even more problematic question emerges. How would you pay nodes without taking money away from mining/staking? A blockchain only has a certain amount of money to spend on its infrastructure (total collectable transaction fees + newly mintable coins). Money spent on the peer to peer network would be money directly taken away from miners/stakers. If you take away money away from the mining/staking reward to pay p2p nodes, it means that there is less money to incentivise mining/staking. If there is less money to collect from mining, then less people will mine, securing 51% of the hashpower will be easier, and you would have defunded the security mechanism of your blockchain. The security mechanism is important because it makes attacking the network costly (which we want, because there’s no point in using a chain which gets attacked). In economics, we call this situation a budget constraint problem. In the blockchain space, we can call it the foundation of the blockchain trilemma.
So, non-mining nodes are volunteers. The peer to peer network exists as it does today because regular individuals decide to set up nodes, voluntarily. So why do people go through the effort and cost of running nodes at all? Well, nodes can broadcast and verify their own transactions — and this is a reason why someone might set up a node (to use the network trustlessly). But in reality, most blockchain users don’t actually care about doing this, so are happy to just free ride off the volunteer p2p network without contributing to it themselves. The main reason people run a node is because they want to contribute to the decentralisation (and subsequent security) of the network — because they want the chain to succeed for moral, political, financial, personal or other reasons. The dominant strategy for creating a strong p2p network in our current paradigm is to cultivate a culture of voluntary node running. To date, only Bitcoin has really succeeded in doing this, for reasons we will discuss shortly.
We can sum up the volunteer problem very succinctly:
Blockchains don’t pay for critical network infrastructure, only mining/staking. Critical infrastructure (such as collecting, verifying and sharing transactions; storing the blockchain history) is provided by volunteers.
The volunteer problem can also be summed up as ‘the free rider’ problem of blockchains, as it is essentially network participants free riding on the work done by volunteers. Most of the subsequent problems we will discuss are a result of this fundamental issue — although they are not usually portrayed as such.
The Scalability Problem
Two key parameters in a blockchain are the ‘block size’ and the ‘block time’. The block time is how often a block is produced and added to the chain. The block size is the amount of data which can be put into a block. Together, these two parameters determine the rate of growth of the chain — the rate at which the size of the blockchain grows.
- Rate of growth of chain (mbps) = block time (seconds)* block size (mb)
In simple terms, the ‘size’ of a blockchain is how much space it takes up on a hard drive — and this depends on both how fast blocks are added to the chain, and how much data each block holds.
Remember when I said that blockchains which process more data (which are more scalable) will have greater storage requirements for full nodes? — Well, here’s the problem with that. Blockchains which make their chain’s rate of growth too high (e.g. through increasing the block size) end up growing the chain size to an unsustainable level, where it becomes unfeasible for volunteers to run nodes because data storage costs grow too high. This leads to a deterioration of the peer to peer network and with it, all of the valuable and network-critical functions which it performs. In short: scaling too aggressively destroys the chain’s infrastructure. Since we do not pay for data storage, scaling a blockchain too much will lead to the chain buckling under its own weight, with a deterioration of the p2p network and all their related functions. This is called ‘blockchain bloat’.
The basic problem is this: attempting to scale beyond the capacities of what volunteers can support will lead to volunteers dropping off the network. Volunteers are nice people, but even they have their limits. When costs to volunteers become too great, they will stop supporting the network. Their costs come in the form of hardware requirements:
“The decentralization of a blockchain network is determined by the ability of the weakest node in the network to verify the blockchain and hold its state. Therefore, the costs to run a node (hardware, bandwidth, and storage) should be kept as low as possible to enable as many individuals as possible to become permissionless participants in the trustless network.
- Starkware, Redefining Scalability
This is where the decentralisation — scalability tradeoff in Vitalik’s ‘blockchain trilemma’ comes from. And is likely the reason why back in 2010, Satoshi somewhat covertly introduced a hard-cap to Bitcoin’s blocksize of 1mb.
While we’re here, let’s talk about a little thing that happened called the blocksize wars. Back in 2015–17, there was a raging debate which tore up the Bitcoin community. The debate was about what the block size should be for Bitcoin. Should it increase its block size to have large blocks so it can be a highly scalable network, or should it have small blocks and be a highly decentralised and secure network — and leave scaling to off-chain solutions?
Remember when I said I’d explain later why Bitcoin (BTC) was the only chain able to really create a strong culture of node-running? This is because at the end of the blocksize wars, only a minor block size increase was enacted (from 1mb to 2–4mb with segwit), and BTC went down the path of small blocks. It severely constrained its scalability in favour of creating conditions favourable to node-runners, to ensure the chain’s long-term sustainability. Hard forks like BSC and BSV were (are?) attempts to steer the Bitcoin protocol down the big-block path. But the small block size of BTC (an intentional scalability cap) makes running a Bitcoin node vastly cheaper than running a node on other chains. This block size cap, coupled with Bitcoin’s rich history, is what has led to the strongest culture of user node-running in the space.
There’s a gold-mine (pun intended) of topics to discuss concerning the blocksize wars, but ultimately it comes down to the simple tradeoff between scalability and the peer-to-peer network.
‘Why is Bitcoin so slow?’, so many complain. There is nothing about Bitcoin that necessitates it being slow. The only reason that Bitcoin can’t scale is because scalability would undermine the volunteer peer to peer network by increasing data storage costs to the point where volunteer nodes would be unable to continue running the network infrastructure. Bitcoin doesn’t scale because it chooses not to scale — not because there is something wrong with its protocol design.
Volunteer networks can only achieve volunteer scale. The Red Cross has been providing volunteer-powered healthcare for over 100 years. But it is not able to scale to the level that it can provide an entire healthcare system for society. As we will circle back to later, in order to provide (produce) something at a scale greater than what volunteers can manage, the market must be rewarded for producing it. Incentives steer the course of production.
“I suspect we need a better incentive for users to run nodes instead of relying solely on altruism.” — Satoshi Nakomoto
*Whether or not one believes that the above quote is from Satoshi comes down to whether one believes Satoshi’s email was hacked or not. Obviously, this is hotly debated — as both sides on the blocksize debate want Satoshi on their side. In spite of all this, I think the point raised in the quote still deserves attention.
The Data Storage/Pricing Problem
At the root of the scaling problem is the fact that blockchains don’t pay for data storage incurred by volunteer nodes. In every blockchain, users pay money today to put data onto the chain forever. They pay a one-off fee to miners, which fluctuates depending on network traffic. The intersection of the supply of and demand for blockspace on that chain determines the fee. Not factored into this fee is the cost of storing the data inside that transaction which archive nodes will have to incur in perpetuity, and which non-archive nodes will have to store for a good while. Not that the fee goes to nodes, anyway (it goes to miners/stakers). This is the case for every transaction on the network. Data storage costs simply aren’t accounted for in the fee pricing mechanisms of blockchains. Nor are the bandwidth costs associated with collecting and sharing transactions, mind you. And as discussed earlier, there is also a significant challenge in actually determining a market price for data storage endogenously (within the network), even if there was a way to pay for data storage without defunding consensus.
Over time, more and more data gets added to your favourite chain without its storage being paid for. The blockchain grows in length. Even on a chain which limits its scalability. This puts increasing amounts of strain on archive and full nodes, who incur the cost of storing network data without direct monetary compensation. The hardware and maintenance costs related to data storage simply snowball over time.
Because there is no compensation for storing blockchain data in consensus, only those who can find a viable business model outside of the blockchain (or those who are extremely militant maxis with sufficient resources) end up running archive nodes for networks. Typically, this is done by a small number of large private (centralised) companies like cryptocurrency exchanges, as well as those who run block explorers who require archive nodes to provide their service. That said, one only needs to look at BSV to realise that nodes run by private companies might not even be profitable if chain scalability is too high, because the chain doesn’t compensate them for data storage costs.
In my opinion, we would all prefer a network which endogenously pays for everything it needs to survive rather than relying on the existence external markets to keep it alive, so that we have a completely self sufficient system.
The Scalability Problem revealed that we cannot scale the network beyond whatever volunteers can manage without sacrificing decentralisation.
The Data Storage/Pricing Problem reveals why: there is a fundamental issue with regards to not paying for data storage on blockchains, which is a problem no matter what the throughput of the underlying blockchain is. In economics, this unpriced impact is called an ‘externality’. The data storage/pricing problem is what causes the scalability problem, so the two may be lumped together as the same problem under either name. It was only for the purposes of education (and because it will be useful later) that I separated the two.
The inevitable outcome of the data storage problem, where an ever-increasing chain size puts cost-pressures on nodes who store the data, is sometimes referred to as ‘blockchain bloat’. Blockchain bloat will be a problem on any chain after enough time (because blockchains only ever grow in size) — though chains which have higher scalability will face it more severely, more quickly. Economists call a situation like this — where the overuse of a public resource (the blockchain) today degrades it for everybody else tomorrow — a tragedy of the commons.
It is worth noting something about Bitcoin here. The Bitcoin (BTC) blocksize is small enough that the Bitcoin chain grows so slowly, there’s a case to be made that increases in data storage costs for nodes might be offset by decreases in the price of hardware as technology improves over time. I’m not aware of any other chain that can reasonably make this claim. So one could consider Bitcoin exempt from the Data Storage Problem (we could also debate this). But Bitcoin is only exempt from the data storage problem precisely because it has decided not to be able to scale its base layer. So a fundamental trade-off has been made. For Bitcoin, The price of avoiding blockchain bloat is 7 transactions per second. So while one problem is solved, another becomes worse.
Above is a chart of the size of Bitcoin’s full blockchain (in GB) over the past five years. You can see that it is not exponentially increasing. It is currently 417GB. A regular person could feasibly run an archival node. But as you move further away from Bitcoin, it becomes apparent that this is not the norm in the space.
Because of its higher throughput than Bitcoin (more transactions per second, more data per transaction, larger state storage requirements), Ethereum’s chain size is growing at an extremely rapid and unsustainable rate. An archive node on Ethereum took up 4 terabytes of space in April 8th of 2020. That is a LOT. More than a normal person is willing and able to run. Approximately, two years later, at the time of writing this article (July 22nd, 2022), an Ethereum archive node takes up 11.3 terabytes of space. That’s more than double as much. And remember, the blockchain is only going to get bigger.
That’s archive nodes, who store much-valued historical data. But what about full nodes, which make up the bulk of the p2p network? Instead of the full history of the blockchain, full nodes (also referred to as pruned full nodes) only store the current state of the chain, including enough historical data to process any new transactions. They verify all new network transactions, but prune (delete) older states from the chain which are not needed to determine the current state (determined by a largely subjective valuation of how much is not risky to delete).
The storage requirements for Ethereum’s two major full node clients over the past four years are shown below. The sudden drop in storage requirements in January of 2022 for the GETH node client was due to pruning more data from the chain, likely because storage costs were rising too much, too fast. This is not a systematic way to handle data-storage. The trend of data storage on Ethereum is clearly up, and at an increasing rate. In the past four years, the storage space of a full Ethereum node have increased from 163GB to 740GB — an approximately 4.5x increase.
I think is fair to say that without any changes to the protocol, if Ethereum’s blockchain continues to grow at this rate, the average user would certainly be unable to run a full node within the decade. Node running would be left to those private companies which are able profit from running nodes (like blockchain explorers) if there are any, and the hardcore maximalists who can afford to do so. The long-term decentralisation of the network would be threatened by the data storage problem.
If you’re an Ethereum maxi, I can already hear you seething. Don’t worry! In another article I will cover Ethereum’s plans to tackle the issue of increasing blockchain bloat.
And I’m not trying to make Ethereum look bad or anything, because it’s well known that Ethereum is vastly more decentralised (and scaling less rapidly) than most of its competitors. Ethereum just made different tradeoffs to Bitcoin. Chains like Solana are growing at even faster rates, and are now offloading the burden of data storage costs to projects like Arweave. In 2021 Solana was growing at 2TB per year, and in 2022 it is now growing at 30TB per year. Its strategy is to have full nodes only store 2 epochs (2 days) worth of recent data, and for archive nodes to use Arweave to store the rest of the chain. This is all just classic blockchain trilemma tradeoff stuff. — And don’t worry, we’ll discuss data storage platforms like Arweave and Filecoin a bit later on in another article.
In short: blockchains exhibit a tragedy of the commons situation with regards to the storing of data. Everybody adds data to the chain, but nobody pays for it — and the cost burden is borne by the volunteer p2p network. Over time, this degrades the security of the blockchain as decentralisation is eroded.
The Network Closure Problem (Background)
This next problem is a scarcely discussed, and often misunderstood one in the space because it is an economic problem, rather than a technical one. It is one that Moxie Marlinspike’s viral blog post, My First Impressions of Web3 indirectly touched on. That article will be useful pre-reading if you don’t understand what something like Infura is, or how a web-wallet like Metamask interacts with the underlying blockchain.
If the concept of the ‘networking layer’ of a blockchain is new to you, and you want to learn how it works on a deeper level but don’t know where to start, then you should then read Preethi Kasireddy’s beautiful overview, The Architecture of a Web 3.0 application. Feel free to keep on reading here if you are already familiar with these things — or if you just want the ‘short’ version. But I highly recommend you revisit those articles at some point if you want to learn more.
I will mention that when I say ‘networking layer’, I am referring to the p2p network which connects users to the blockchain.
In order to get to the heart of this problem, we are going to learn some economics — and what Satoshi really invented in 2008. At first you might think that what I’m about to discuss isn’t relevant, but it’s actually the basis of understanding the entire value proposition of blockchains and cryptocurrencies — so strap in and enjoy the ride.
Public Versus Private Goods
In economics, a public good is something that has two properties:
- It is non-excludable: nobody can stop somebody else from using it
- It is non-rivalrous: one person using it doesn’t stop other people from using it
A public park provided by the government is a good example of a public good. Nobody can stop you using it, and using it does not stop other people from also using it.
A private good is the opposite of a public good — it is something that is excludable and rivalrous. Something like a burrito is an example of a private good — its owner can stop others using it, and its usage or consumption stops others from using or consuming it.
A point of confusion is often that ‘public’, in the context of economics, means’ government. A ‘publicly produced good’ would refer to a good produced by the government — whereas a ‘privately produced good’ is something produced by the private sector (individuals, businesses). A ‘public good’, however, can technically be produced by the private or public (government) sector. It just so happens that, for reasons we will now discuss, inducing the private sector to provide public goods is very difficult — and so the government is basically always the provider and producer of public goods.
An Economics Lesson on Public Goods Provision
Normally, the government pays for public goods using money collected through taxation or monetary debasement. Why? Because there is no economic incentive for the private sector (you and me) to produce them.
I’m going to explain exactly what I mean by this using the example of trying to provide a public park.
Would you (the private sector) voluntarily pay the cost of building and perpetually maintaining a park that is open for everybody to use? It would be a constant money drain, and you would not get compensated for it. Meaning, if you didn’t have some way to make enough money to pay for it, you’d eventually go bankrupt.
Maybe if the park was small enough, you might volunteer to produce it for your community if you had the money. But in this situation, the park’s provision would not be ensured into the future. If you died, it would pile up with litter and damages with nobody being paid to constantly keep it clean or maintain it.
What if the public park was supported by the altruistic donations and efforts of the community who would use it? Well — first, as we learned earlier, volunteers can only provide volunteer scale. If we rely on volunteers, then the scale of the public good will be constrained by what volunteers — who lose money and effort to fund and care for the public good — are able to handle.
Secondly, we have an incentive problem on our hands. Suppose the (small) park is paid or cared for by a community of 20 volunteers. If I’m in the community, and I know that 19 other people are also paying or caring for the park voluntarily, then I’ll just stop paying for it and let them pay for it. They can do the work, and I’ll still get the benefit. You might object: “but that would be mean / wrong / unethical / bad!” — Oh well. I’m not going to pay for it. And because it’s a public good, I can still use it just as much as anyone else, and nobody can stop me. This is called free riding, and it is something we observe all the time in the real world. A classic example is free riding on the work produced by a hard-working member of a group assignment and doing nothing yourself. Or not paying for your bus ticket, while everybody else does.
I probably won’t be the only one with this free-riding idea either. One by one, other volunteers will catch on and also stop paying or caring for the park — because they don’t need to pay for it to use it, and it is still getting taken care of by other people. This is prototypical game theory — see here for more. Individuals act — not groups. And individuals do what is in their personal best interests. So the incentives on the individual layer determine the outcome we get. Even if everybody wants the park in theory. This is the essence of the Prisoner’s Dilemma in game theory.
You might say: “but we can stop this from happening — the volunteers who are paying or caring for the park can just ban those who aren’t from entering the park, or make non-contributors pay to use it!” These are pretty good ideas, but what you’ll get if you implement them is no longer a public good, because this introduces privatisation and therefore excludability. Those paying would have a monopoly on who could or couldn’t access the park. Note here, that introducing privatisation and subsequently excludability (being able to stop people using the park) actually enables a business model where you could allow park access to those who paid you for it. This would be a perfectly viable business-model which would ensure the sustainability of the park’s provision (assuming there was market demand for it). But we no longer have a public good.
Okay. We can’t ban people or make them pay, because then it wouldn’t be a public park. But “surely”, you might say, “because people value the park, and having the park gives them some kind of value, there will be people who are willing to pay for it. Even if they are volunteers, they will still get ‘compensated’ — just not with money!” Well, like we said before, at a small enough scale altruistic and self-sacrificing volunteers might be willing to take the economic hit to provide the public good which they value. But this is not a scalable approach. How can I prove it’s not scalable? Because we don’t get the private sector providing healthcare, education, roads, bridges, parks and infrastructure for free at nationwide or global scale for non-monetary (e.g. moral) utility! And even if there was a willing volunteer who had a LOT of money and could afford to produce a public good at a societal scale (insert your favourite company/billionaire here), they would not be able to do so sustainably. Eventually, the perpetual costs of providing the public good’s expenses would simply bankrupt them. And even if it was magically only a one-off cost to create and maintain the public good, there would still be no monetary return on the investment — which, compared to alternate uses of the money, would seem wasteful.
People not familiar with market failures often reason: “if [public good X] is really valuable to a lot of people, then someone will pay for it.” But that isn’t how markets work. The market will only provide something through the introduction of closure. However, by now it should be clear that the free market will not provide public goods unless they privatise the resource in question, so that they can justify its production costs through profit. That’s the only way we can incentivise them to produce the good for us. But privatisation leads us to lose the property of non-excludability — which is why we can’t all just go and relax in a private golf club’s park. It is not a public good.
This situation, often deemed a ‘market failure’, is why we try to get governments to provide public goods for us using taxation (whether directly or through inflation). Notwithstanding the point that government failures are also a very real possibility, both market and government solutions to the public goods problem introduce a trusted third party for provision.
What Type of Good is a Blockchain?
The private sector will not produce a public good unless they are able to remove openness (non-excludability) and introduce closure (excludability) through privatisation.
But what if there was a special kind of public good which actually paid the private sector to produce itself, while maintaining its core properties? What? How is that possible?
Blockchains are not ‘owned’ by anyone. It’s one of the weirdest things about them. If they are built right and the network becomes sufficiently decentralised (if they are good blockchains), no single party owns or can unilaterally control them. They are non-excludable, because anybody can use them — and they are also non-rival because one person using them does not stop others from being able to use them (your transaction will get onto the chain). They are public goods, in the purest sense of the term.
In fact, nearly everything desirable about something like Bitcoin comes from the fact that it is a public good. Censorship resistance is the inability for anyone to halt or tamper with your transactions on the network. Openness, that anyone can use the network and that no-one can be excluded from it, is another highly desirable property of blockchains. Blockchains (when built right) are public goods, and this is exactly why we like them!
Except, unlike basically every other public good, blockchains are not provided by the government. They are provided by the private sector. Blockchains are privately produced public goods. A good blockchain is a public good which incentivises its own provision.
Blockchains are privately produced public goods. A good blockchain is a public good which incentivises its own provision.
This is possible because every blockchain is powered by a cryptocurrency which can be used on its network. If you just asked the private sector (you, me, companies) to maintain a blockchain — a distributed ledger — they would say no, because they have no incentive to do that. Just like with a public park. But if you told the private sector that if they provided the blockchain, they would get paid with a currency that could be used on the blockchain — then they would happily provide the necessary network infrastructure, supposing they thought the currency would have sufficient value (utility) to others and the network would grow over time.
This was part of the genius of Satoshi. He made a network that pays for its own existence. A network which funds itself. A network which is self sufficient. And not is it only self-sufficient, but it is also a public good which is self sufficient. Something open that nobody can be excluded from using or participating in, and which me using does not stop you using. This was one of the key revolutionary steps of Bitcoin. It was a self-funding public good that did not rely on a closed set of participants to provide. It is not enough that a blockchain does what we want it to now. We need it to be able to survive into the future — indefinitely. Otherwise what we have is merely a temporary chain, doomed to inevitably lose its core properties in an attempt to stay alive.
As we saw before, the private sector can only viably provide public goods when they introduce closure — when they take ownership of the resource or good in question. This was the standard theory, anyway. Yet, with Bitcoin, this was not the case. Nobody owns Bitcoin. It is a public network.
Blockchains being public networks is the reason why ‘attacking the network’ means exerting control over the network.
An important point needs to be made here. Why did I say that blockchains are only public goods “when they are built right”, or that only “good blockchains” are public goods? The reason is that slapping the word ‘blockchain’ onto some code does not mean you have created anything of value. When Satoshi created Bitcoin, he created something new. Something fundamentally different to most of what passes for a ‘blockchain’ today. He created a self-sustaining and secure public good.
The key properties which make a blockchain valuable in the first place are:
- Openness (non-excludability from both consensus and usage)
- Self-sufficiency (incentivises the provision of its own infrastructure)
- Censorship resistance (prohibitively expensive to tamper with or cheat)
I’m going to call these three properties The Satoshi Properties — as they constitute the core innovations made by Satoshi Nakomoto. Without them, blockchains are worthless. Trustlessness and the disintermediation of third parties would not be possible. All other properties (e.g. decentralisation) are only valued insofar as they help to attain the above properties. If they did not contribute to the attainment or preservation of the above properties, we wouldn’t really want them. If you don’t believe this is the case — try to imagine a blockchain without these properties, and you’ll quickly find that it doesn’t require a blockchain to exist.
What really makes a blockchain desirable in the first place? Self-Sufficiency, Openness and Censorship Resistance. No matter HOW decentralised or scalable, you would not want to USE a blockchain without these properties. The Satoshi Properties are the real blockchain trilemma.
The Network Closure Problem
As we have learned, the networking layer (the p2p network) is a key part of a blockchain. It is crucial to its proper functioning and for the long-term maintenance of the Satoshi Properties.
But the networking layers of blockchains are not paid for by proof of work and proof of stake consensus mechanisms. Which means that only part of the public good of the blockchain is being self-funded. Which means that it is left to volunteers or private entities to provision the network layer.
From our discussion on public goods, we now know that this leaves only two possible options. Either:
- Don’t scale the public good, in this case the networking layer, so that volunteers can provide it for free — OR
- Let the private sector come in and fund the provision of the good, at the cost of privatisation and the loss of non-excludability
And this is exactly what the crypto space’s solutions to the problem are! Bitcoin decided not to scale (7tps) and let volunteers run the p2p network — and other chains decide to scale more, but let private companies come in and fund the network infrastructure.
In the case of Bitcoin, we don’t get commercial scale (volunteers only provide volunteer scale). In the case of other chains, what we see is that the network layer begins to get monopolised or cartelised by private companies who occupy positions of tremendous power on the network. Moreover, these private companies themselves run on totally centralised, also privately owned companies like AWS. Also note than in addition to centralisation posing a censorship risk, it also poses the risk of being more easily hacked or disrupted. This all brings us to the network closure problem.
Let’s talk about the prototypical private company which has emerged to replace a blockchain’s volunteer peer to peer network.
Infura is a node-providing company for the Ethereum blockchain. Remember that name. They are a private company which sets up nodes for people like developers, to connect end-users (you) to applications on the underlying blockchain — and route transactions they collect across the p2p network.
Here’s a quick overview of what Infura does from their FAQ’s:
“Blockchain applications need connections to peer-to-peer networks which can require long initialization times. It can take hours or days to sync a node with the Ethereum blockchain and can use more bandwidth and storage than you had planned.
It can get expensive to store the full Ethereum blockchain and these costs will scale as you add more nodes to expand your infrastructure. As your infrastructure becomes more complex, you may need full-time site reliability engineers and DevOps teams to help you maintain it.
Infura solves all of these problems by providing infrastructure and tools that make it quick, easy and cost-effective for developers to connect to Ethereum and IPFS and start building awesome decentralized applications.”
Essentially, the busier a dapp, the more transactions need to be processed, the more and stronger nodes are needed to manage the increased data flows for that dapp. By providing and running node infrastructure for Ethereum developers, Infura covers a massive cost (money and time), processing up to 100,000 requests per day for free.
All this ‘free’ or ‘free-market provided’ infrastructure sounds great in theory, but it actually comes at a massive cost.
Firstly, there is a centralisation risk. Since Infura exists solely to spin up Ethereum nodes, they experience the benefits of economies of scale — meaning, it is cheaper for them to run nodes than smaller parties running nodes. Obviously, this has led to massive centralising pressures. As Google and Amazon (web2 analogues to Infura) show us, IT industries tend towards monopoly. And what do we see with Infura? Infura nodes process 80–90% of all transaction flows on the Ethereum network. This is an extremely centralised system. Infura also owns and runs Metamask, which is how basically everybody connects to the Ethereum blockchain.
Infura runs so much of the Ethereum network that when they forgot to update a bunch of their nodes in 2020 — they unintentionally hard forked Ethereum (!) and crashed Metamask. What’s more, in 2022, Infura accidentally blocked the entire country of Venezuela and regions of Ukraine from using Ethereum while trying to comply with government sanctions (see here and here), also causing massive network slowdowns. More recently, Infura (and their largest competitor, Alchemy) blocked access to Tornado Cash to comply with government regulations. Since Infura is a monopoly and the problem of provisioning of an open network is not regulated by consensus, there is no punishment for this censorship. In classic web2 fashion, Infura also collects your data, too.
I’ll also mention that Infura itself (the AWS of Ethereum) runs on Amazon’s AWS — so we have a centralised company running the blockchain by using another centralised company. In 2019, 25% of Ethereum’s nodes already ran through Amazon (including Infura). Oh, and even Amazon themselves want a piece of the game of running Ethereum nodes now. All this centralisation poses what is called a ‘redundancy’ risk. Locating so much of the network infrastructure in one place increases the network’s vulnerability to hacks, political pressure, regulation, natural disasters, power outages and other disruptive pressures.
The reason the networking layers of Ethereum is mostly run by large businesses subject to regulatory capture is because volunteers can’t afford to sustain the scale the blockchains is trying to process. This means that only for-profit firms can afford to run the network. Companies like Infura and Alchemy monopolising the network layer are merely symptoms of the volunteer problem — the fact that blockchains do not pay for network-layer infrastructure out of consensus.
Due to the volunteer problem, a mere handful of privately owned and run companies are running the second largest crypto network on the planet — and there is nothing stopping them from abusing their position of power. Hoping that the private companies who decide to run blockchain infrastructure will not bend to political whims or exert their own control over the network undermines the virtue of trustlessness which the entire space was founded upon.
There are some competitors to Infura — such as Alchemy. But if Infura suddenly broke or got hacked, Alchemy would not have the capacity to handle all of the network traffic which Infura is processing, because Infura is so much larger. However, even if Alchemy did have this capacity, it does not fix the problems created by allowing the private sector to run your networking layer.
There is a second, still deeper problem here.
The REAL Network Closure problem is not merely about centralisation. Put simply, the network closure problem is that the business models needed for the private sector to provide the p2p network undermine consensus openness (universal participation on equal terms).
Because the private sector cannot afford to run nodes for everybody at a loss indefinitely, they have to monetise the transactions they collect in order to sustain their business. Monetising transaction flows just means using the transactions they collect to make money — normally by taking part of their fees. An example of how this would happen is if Infura either set up their own staking/mining nodes and only routed transactions to themselves — or if they partnered with existing miners/stakers and agreed to route transactions exclusively to them for a portion of the transaction fees.
Here’s the problem though: if user transactions represent profit for private infrastructure providers like Infura, then these companies will have no incentive to share them with others on the network — which puts them in a position to determine who can participate in the network and at what rate of profitability. They will have an incentive to collect the transactions, because they can make money off them, but a dis-incentive to share the transactions freely with all network participants, because otherwise somebody else could get the fees without giving them a cut. There would be no recourse or cost to a company like Infura should they decide to do this, or engage in any other exclusionary behaviours.
Although there are certainly issues regarding economies of scale and single points of failure, network closure is ultimately an entirely distinct matter to centralisation. Whether there is one or ten firms providing the networking layer of the blockchain, since they are not paid out of consensus they all must monetise transactions, and so they all must introduce closure around network data flows. It is necessary so that they do not go bankrupt.
This forces these companies into de-facto positions of power to determine who gets those transactions (who makes money off of them) and who doesn’t. They will either forward transactions they collect to their own mining/staking nodes, or miners/stakers who they are partnered with. That way, they can take a cut. Consequently, if a network decides to scale and leave network infrastructure to the free market, as private firms fill the void left by volunteers they will come to determine who is able to participate in consensus. The property of openness gets destroyed.
At no point do Infura-like businesses have an incentive to share transactions they collect with the rest of the miners or stakers on the network, because they would not get paid for doing so. If they did share transactions freely, they would be doing costly work for free, and the rest of the network (including their competitors) would free-ride off of their work. This is not a viable strategy for a private company. Like the volunteers before them, free-riding pressures would build and the company which decided not to monetise transactions would be unable to bear their infrastructure costs at scale. Hence, relying on external markets to provide the networking layer (a public good) at scale necessarily leads to closure around data flows and the loss of public-good properties — because closure is the only way businesses can be incentivised to provide the good.
“Businesses will process transactions in exchange for a part of the transaction fee, which requires adding closure to data flows, which cannibalizes the openness of consensus-layer data-flows. This kills the Network”
“The existence of private firms does not necessarily lead to monopolies (although it seems prevalent with high-tech businesses, so this isn’t necessarily wrong). The inevitability of closure is because private firms are now collecting money for the consensus layer — which is another public good.
Consensus requires permissionless access to the payment faucet to remain open/non-excludable/egalitarian. Firms on the network level can be free-ridden on by other firms on the network level [if they share transactions], based merely on their altruistic support for the underlying consensus layer.
And firms maximize income by cannibalizing those revenue flows. So transaction monetization becomes the norm, as with TAAL. The interior consensus-layer public good poisons and infects the incentive structure of the exterior market unless the external firms respond by closing access to the value they provide it.”
— David Lancashire
If Infura collects 90% of network transactions, unless they want to run at a loss, they will decide who gets the fees (who has the right to make money) and inevitably push other participants not privy to their fee-flow off the network over time. This is a closed system. Like the banking system. Which is the opposite of what Satoshi invented. In the banking system, banks take your transaction fees and provide you with transaction-processing services — but they do not share your transactions with other banks for free. Instead, they introduce closure so that they can make a profit and continue providing the service. It’s not that Infura is evil — they’re not at all. It’s just that they need a business model to survive, and this business model contradicts the value of openness (that anyone can participate on equal terms) in the blockchain space.
The problem here is very simple: if we wanted privatisation of data flows on the internet, we would just use web2. Companies like Google already do this for us, and the cost is closure: them having the powers of censorship and excludability.
This is the exact same reason Amazon, Google or Facebook are not open. They need to pay for the infrastructure, so have to monetise and take ownership of data flows. Which puts them in a position of power to exclude others from those data flows. Amazon, Google and Facebook are not, however, attempting to provide public goods. It all comes down to the fact that providing a non-excludable good can only be done by volunteers at a loss, or by the private sector introducing closure (excluding others).
The genius of Satoshi was figuring out how to endogenously pay for network security and induce the private sector to provide something with public good properties in an open manner (in a way where everyone can participate equally). But even Satoshi was not able to figure out how to pay the networking layer of cryptocurrencies, which is why Bitcoin’s network is run by a culture of volunteers. For chains which do not constrain their scalability, the abandoning of Section 5.1 of the Bitcoin whitepaper is inevitable when private firms who need to monetise transactions to cover costs step in to provide the networking layer. The ability to limit who can participate on the network profitably can even be seen as a type of discouragement attack.
Note: Some people suggest that lite clients will fix the Infura problem, but forget that lite clients ultimately still rely on full nodes run by companies like Infura (the only ones who can afford to run full nodes at the scale the chain requires) to process their transactions. More lite clients means higher costs (e.g. SPV generation costs) to the remaining full nodes who need to service lite blocks.
I’ve used Ethereum as my main example here, but it needs to be understood that this is not just an Ethereum problem. Like the data storage problem, every chain which decides to scale has or will have this problem. The network closure problem is probably the most overlooked, ignored and hand-waved away problems in the entire crypto space. But maybe that’s just because nobody knows how to fix it.
Block Subsidies and Network Security
Blockchains pay miners and stakers using some combination of fees and newly minted coins (monetary inflation). Bitcoin, for example, uses inflation and transaction fees to pay for its security. Because it has a hard cap of 21 million Bitcoins, eventually it will stop using inflation and pay miners solely with transaction fees. This means that Bitcoin’s security budget will eventually stop being subsidised by newly minted Bitcoins, and will shrink to just the size of the fee market.
If the price of Bitcoin does not sufficiently appreciate to make the value of transaction fees compensate for the lost block rewards each halving, many miners will be pushed into economic losses and leave the network. Less miners means the total hashrate falls. Since Bitcoin’s security (cost of attack) at any given point is a function of the total hashpower of the network, this means attacking the network becomes cheaper. In Proof of Work, the cost of attacking the network is 51% of the total hashpower. (Likewise, it is 51% of the stake in Proof of Stake)
Compared to the newly minted coins in each block (currently 6.25 BTC per block produced), transaction fees only make up a miniscule amount of the block reward which goes to miners. In fact, due to increased usage of the lightning network, Bitcoin fees are not scaling with adoption — and the main incentive for mining is still without a doubt the block reward. This poses a long-term problem with regards to Bitcoin’s security, which will eventually only be funded by fees. And the same goes for any hard-capped crypto asset.
To learn more about this issue from a variety of perspectives, I recommend reading the following:
- The security trilemma and the future of Bitcoin (article)
- A model for Bitcoin’s security and the declining block subsidy (paper)
- On the instability of Bitcoin without the block reward (paper)
- Bitcoin’s security is fine (article)
One solution to the problem of the disappearing block subsidy is to make the chain’s native currency permanently inflationary — to commit to relying on the block subsidy indefinitely. The problem with this is that it destroys the monetary premium associated with the asset, which itself can pose security risks.
Ethereum’s solution is to make Ether inflationary, but also burn enough of every transaction fee that it outweighs the inflation, and ETH becomes net deflationary. This manages to preserve the deflationary aspect of the currency while not relying on fees for security — but (to the best of my knowledge) at the cost of higher fees than if we didn’t have to burn part of each fee to achieve deflation.
For now, we can leave this problem at: the fixed supply monetary policy which Bitcoin champions — and which this space was founded upon — poses a threat to the security and long-run self-sufficiency of any chains which employ it.
Majoritarian Attacks
Majoritarian attacks, also known as 51% attacks, are a fundamental attack vector on Proof of Work and Proof of Stake networks. How they work is very simple.
The blockchain is a connected list of sequential blocks. At any given moment, block producers compete to add a new block to the ‘tip’ (the front, the newest end) of the chain. As mentioned earlier in the article, the tip of the chain can temporarily fork when two block producers create a block at the same time and both propagate their new block around the network. In this situation, the network will have two copies of the blockchain with different blocks at the tip, and doesn’t know how to decide which one to accept as reality.
The way the network resolves this split is by allowing the network participants to continue producing blocks freely on either of the chains — and selecting whichever chain is the longest chain as the true blockchain. At some point a miner on one of the chains will produce a block before miners on the other chain, propagate it around to the network, and their newly produced longest chain will be accepted. The block produced on the alternate chain gets deleted (an orphaned block), and everybody adopts the longest chain. Transactions in the orphaned block which did not make it on the longest chain will return to mempools. The odds of producing two blocks again and again, so that the fork is never resolved and two equal length chains are extended together forever, are next to zero.
This fork-resolution mechanism, this consensus mechanism of always adopting the longest chain, is called Nakomoto Consensus. It was another brilliant invention of Satoshi, and most blockchains use it to this day.
The problem which arises as a result of Nakomoto consensus is that if you can acquire a majority of the work required to produce a block (hash or stake), you will be able to consistently create the longest chain and inevitably orphan all blocks which you do not produce on the network.
Even if someone produced a block before you and the network added it to the chain, you could just ignore it and keep mining on your own copy of the chain. Since you have 51% of the work function and produce blocks at a faster rate than the rest of the network, eventually you will produce a longer chain and be able to orphan any other blocks the network produces. This means that you would collect the entirety of the network revenue, fees and block subsidy, in perpetuity. You could then use this money to fund your attack and continue collecting revenue (a fee-recycling/discouragement attack). With the ability to consistently override any alternative chains, you would have the power to decide every block that goes onto the chain. This means you could exclude transactions from ever making it onto the chain and censor transactions, killing the blockchain. This is a 51% attack.
Although 51% attacks would devastate any blockchain, I will note that in Proof of Work blockchains, past blocks are prohibitively costly to orphan because an attacker would have to rehash every subsequent block on the chain after the target block (including ones created during the attack) to create the longest chain.
Proof of Stake tries to achieve a similar property to this with ‘finality’ and slashing, but this comes at the cost of using closed validator sets to establish finality (a pre-Satoshi solution to the Byzantine Generals Problem which does not provide openness) — whereas in Bitcoin confirmation is not final but probabilistic, relying proof of work instead of closed voting rings to maintain cost of attack. The finality methodology can also have some problems at scale due to the need for constant communication between a larger group of network participants to establish finality — and could be subjected to halting attacks.
Discouragement attacks
Above, I briefly mentioned an example of a discouragement attack. In a discouragement attack, an attacker tries to prevent other network participants from receiving income to drive them off the network (discourage them from staying around). This can involve the attacker incurring a loss or lower than average ROI themselves for a period of time to render other actors unprofitable, so that ultimately others leave and they end up with increased power over the network. This would put them in a position to claim a greater share of network revenue — and could even set them up to perform a 51% attack. For example, spending money on hash to attain 51% of the hash to produce the longest chain, orphan all other miners, and then drive others off the network is a discouragement attack — even if initially it costs the attacker to pay for hash.
Although it is an unconventional way of looking at them — discouragement (and arguably, 51%) attacks arise from the fact that blockchains pay for block production. Since the creation of a block is the activity which gets paid, producing a majority of blocks will put you in a position to earn the most money, which you can then use to buy more hash/stake than the rest of the network.
Commodification of the work function on secondary markets
Majoritarian and discouragement attacks are made even more problematic by the fact that the forms of ‘work’ used in Proof of Work and Proof of Stake (hash and stake respectively) inevitably become commodified on secondary markets.
I’ll explain this in simple terms:
- PoW and PoS require miners/stakers to do work (mine/stake) in order to produce a block
- Individuals can mine and stake freely
- Eventually, someone gets the idea that they can rent hashpower or stake to individuals/parties in the broader economy outside of the blockchain (a secondary market)
- Hashpower and Stake become commodified, meaning, treated like commodities. Markets form around buying, selling and renting them.
What is the problem with this? The problem is that if I own 30% of the hashpower of Bitcoin, I can rent another 20%, bear the cost of that for a while, and perform a 51% discouragement attack and eventually make my money back and more. 51% attacks using rented hash have already proven effective on networks with weaker security.
Similarly for stake, I could just rent stake and use that to attack the network. This excellent article details the rise of staking derivatives on Ethereum and how this secondary market for stake poses security risks from concentration of stake. I highly recommend reading it sometime, as it is a serious issue unto itself.
The worst part about the commodification of the work function is that nobody can stop people from renting hash or stake to others, because the renting occurs outside of the blockchain. It is a secondary market activity which the blockchain has no way of penalising, and therefore a seemingly insoluble problem. It’s as if the problem is happening in a higher dimension which the blockchain can’t access.
Miscellaneous Incentive-Based Problems:
There are some other incentive-based problems I wanted to briefly touch on, which I can devote more time to exploring in a future article. Different chains are afflicted with them to different degrees.
One of them is called header-first mining and we see it happening on Bitcoin. It is a significant problem because it shows that the only people who are actually paid anything at all out of consensus are avoiding storing the chain’s data. It is a classic example of an incentive misalignment problem.
Note: a block header is a unique, sort of signature each block has — and is composed of multiple components. It is deemed to be valid when the hash of it meets the requirements to win the proof of work puzzle, which requires brute force mining (spending electricity) to achieve. A block header can be valid even if there are invalid transactions inside the block.
Recall that Proof of Work and Proof of Stake do not pay for transaction validation. They only pay for mining and staking. This generates the problem of header-first mining, where to save space and time, miners only validate block headers — rather than full blocks (the transactions inside blocks). This weaker form of validation can lead to network-dysfunctional situations where they can validate a block with an invalid transaction inside it (see link for more). Header-first mining once led to a 6-block long invalid Bitcoin fork. Again, this problem is resultant from the fact that blockchains don’t pay for transaction validation — only mining/staking.
Another good example of how incentives can drive the behaviour of network participants in directions we don’t want is selfish mining. Again, this is actually a nontrivial security issue which arises due to incentive misalignment. To learn about selfish mining and a couple of other related problems like delayed block propagation clearly and quickly, I recommend this short and extremely sharp article by Aaron Van Wirdum.
Overview of Problems
The following graphic maps out all of the problems discussed in this article.
I hope now you understand what exactly is broken about our blockchains, and why. Again, if you locate the clapping icon👏 on the bottom of your screen and hold down your click button on it — you will be able to give this article up to 50 claps. This would be much appreciated.
A reminder that this article is directly pulled from my larger, four part article Defusing Crypto’s Ticking Time Bomb. Parts III and IV have their own respective articles also, if you would prefer to read them in isolation. In them, we will cover both current solutions to these problems (Part III) — as well as some cutting-edge, non-mainstream solutions to the problems at hand (Part IV).
