Go Fork Yourself (to More Bitcoin Transactions)

Bob McElrath
Apr 19, 2016 · 9 min read

What is Bitcoin? There are two good, possible answers:

  1. Bitcoin is whatever is contained in the longest sha256 proof-of-work chain
  2. Bitcoin is the set of transactions deemed to be valid by the reference implementation of Bitcoin.

As with all great arguments demanding of blood and tears, the source of the argument can be boiled down to a semantic disagreement: the choice of definition of Bitcoin by premise #1 or premise #2 above. Now steel yourself for a hard truth. Arguments on both sides have merit. Both are also profoundly unappealing in their own ways.

Raising the block size is a simple change and reasonable change, but has the unpleasant side effect that old nodes will not see payments made after the change. New money has to be interchangeable (fungible) with old money. The last sentence is basically the definition of a soft-fork. A hard fork means that new money is incompatible with old money. The fear is that let’s say you created a bitcoin-based business, let’s say it’s Satoshi Dice, then you fell ill and were unable to upgrade your bitcoin node. If a hard fork came, your business would cease to function. Let’s call this (hard fork) path violence.

On the other hand, being forced to keep 1MB blocks forever is profoundly ugly, for any software developer. Anyone looking at how bitcoin works in the future will wonder why transactions are divided into a 1MB blob and another blob. Let’s call this (soft fork) path grotesque.

Phrased in terms of questions #1 and #2 above, the answer cannot really be entirely either #1 or #2. It has to be both. The thing we call “Bitcoin” must both follow the highest proof-of-work chain, and that chain needs to be the one validated by all implementations of “bitcoin.” It would cause irreparable harm to the Bitcoin ecosystem if #1 and #2 diverged from a hard-fork.

“For every complex problem, there is a solution that is simple, neat, and wrong.” — H. L. Mencken

I think the way to get capacity increases in a grotesque way (soft forking) is not clear to very many people. Basically, if your node running Satoshi Dice only sees 1XXX… and 3XXX…address formats, then that means your transactions must appear in the usual 1MB block. All your customers must be able to both send and receive funds to your wallet. A soft-forked capacity increase must use a new address format which indicates that your wallet, and the bitcoin nodes it depends on, will see the transactions. A new address format is described, in a soft-forking way, by BIP142. The new addresses starts with p2 and looks like:


While invented for Segregated Witness, this mechanism, in general, signals that a node can understand a new form of transactions. We can re-use this mechanism to soft-fork a capacity increase. This mechanism also uses anyone-can-spend transactions which “hide” the new-form transactions from old nodes, while still allowing old nodes to see the movement of coins.

The mechanism to increase capacity is to have an extension block that contains more transactions. This can really and truly increase the number of transactions. At the same time there are a number of ideas regarding weak blocks sometimes also called “mempool synchronization” that mitigate the problems caused by orphans, and an increasing orphan rate caused by larger blocks.

An example of a braid, from my upcoming paper. The genesis block is on the left (orange) and two chain tips are in the upper right. Successive colors correspond to “cohorts” — an analog of a bitcoin block. This scaling solution has no orphans (stale blocks) and no block time. It is far more scaleable and faster than a blockchain.

Now the exact form that an “extension block” could take is not decided. Simply increasing the block size this way is not going to scale because it increases the orphan risk. So some of us are working on ways to scale the transaction rate by fixing the orphan problem. My braids project is one, but there’s also the leader election mechanism of Bitcoin-NG and the subchains idea of Peter R. Rizun. So the “extension block” might not be a block at all. It might be a braid, a subchain, or a round-robin leader, each of which is an entirely new way to publish transactions that can drastically increase the number of transactions Bitcoin can handle, without resorting to off-chain solutions like payment channels and the Lightning Network.

These solutions are nascent, however, and need more study. With my braids project, I will publish some python code that generates braids, so that everyone can study the incentive structure and figure out a miner incentive structure that removes serious existing problems in bitcoin such as selfish mining. The people working in this direction are some of the most thoughtful people in the Bitcoin ecosystem, including Emin Gün Sirer of Cornell and his post-doc Ittay Eyal, as well as Peter Rizun. The latter has created the new Ledger Journal, to peer-review bitcoin papers. Because we want to create the best system we can, and we need to bring the scientific process to bear on Bitcoin to achieve that.

I work on Bitcoin (as opposed to any alt-coin — and yes I have built one of my own) because that’s where the most intellectual capital exists. I’ve spoken with a great many people in the bitcoin ecosystem, and frankly, Jeff Garzik, Mike Hearn, Gavin Andresen, Peter Wuille, Adam Back, Bram Cohen and Greg Maxwell (among others) have some of the most important contributions to bitcoin and are the best brains around. I’ve spoken with nearly all of them on IRC at one time or another, and my decision to work on bitcoin was largely decided by my personal evaluation that these people are very smart. Let me specifically call out and thank Greg Maxwell and Peter Wuille for patiently explaining and discussing things with me and others on IRC.

My biggest concern is that the community of intelligent people talking to each other will dwindle. We can’t do this alone, we need to talk to each other. There’s a place for a lone genius to work, let’s call him or her Satoshi. But future development depends on us working together and feeding off each other’s insights. Peter Wuille’s recent Segregated Witness project received a boost when Luke-Jr realized how to soft-fork it in. This happened because Peter was careful and deliberate about researching his idea, and he communicated it so Luke-Jr could have his insight. All of these people are committed to improve the capacity of bitcoin. But, I think, have not given a detailed enough roadmap to stave off community fears.

The loss of intellectual capital on Bitcoin is unfortunate. Some time ago we already lost a number of talented people like Vitalik Buterin to the Ethereum project, because of a disagreement as to how to grow and expand the system. However Ethereum’s smart contracts are being brought to bitcoin anyway. Now we’ve got Mike Hearn rage-quitting over a similar perception that it can’t grow. Besides Bitcoin and Ethereum, the other interesting projects in my mind are the privacy-enhancing extensions mostly based on ring signatures. Altcoins based on this technology, however, suffer from a lack of developers, despite being a very good idea. This idea and the importance of anonymity in payments is not going away and new tech based on zero-knowledge proofs is being advanced by the Zerocoin project and ZCash as well as Confidential Transactions in the sidechains project. I’m confident that eventually it will be included in Bitcoin itself, and I encourage those interested in this tech to remain engaged with Bitcoin and work to get it included in Bitcoin itself.

In the middle of this debate, questions of bitcoin “governance” have come up. Governance of software projects has traditionally fallen to organizations like the W3C, ISO, or ECMA. Historically, every new protocol has had a reference implementation during its nascent phase. HTML had WorldWideWeb and Mosaic; Cascading Style Sheets were prototyped in the Arena and Argo browsers; C had a prototype compiler. In as much as Bitcoin is a “protocol” in the same sense as these other things, it has a reference implementation in the code that Satoshi produced, which has evolved into a great number of derivations of that reference implementation. Many of those parse and validate the existing bitcoin blockchain. But the history of software development shows that the reference implementation rarely survives. I bet you’re using Chrome, Firefox or Safari and not Mosaic, aren’t you?

The reference implementation rarely survives. What tends to happen is that with time, wisdom is gained in how the protocol works and how to build it better. At some point, a clean-room implementation is almost always better than starting from the reference implementation. In the Bitcoin world, we have have at lest five re-implementations: libbitcoin, BTCD, Bitcore, and then there’s Bitcoinj by Mike Hearn, and Picocoin by Jeff Garzik. I find it interesting that two of the camp who are in favor of a hard fork are people who have done clean-room implementations of bitcoin. I presume that they are people who appreciate good clean software design, and the hard fork is simple. I assume they think my proposal above is unnecessarily complex. I’d agree.

In my experience in theoretical physics, it is fairly common practice to re-do someone else’s calculation. It’s also just good science to re-do it, and re-do their experiments as well. It’s a learning process. Re-doing their calculation or re-implementing their software is a great way to figure out how it works, and in the process you’re likely to re-implement it a little better than the original. You have the benefit of hindsight! You can steal the good algorithms where the original author did something good, and use better ones where your knowledge exceeds that of the original author!

The desire for clean software engineering is a natural one. In physics we have a luminary, Paul Dirac who once said:

“This result is too beautiful to be false; it is more important to have beauty in one’s equations than to have them fit experiment.” — Paul Dirac

This takes the desire for beauty, or analogously cleanliness in software design to an extreme. I think this is a step too far, but Dirac’s search for beauty was fruitful. He gave us the equation of motion for the electron, among many other very important contributions. There is value in seeking the simplest solutions, but there’s also great value in not breaking money. If you have an Intel or AMD processor in your computer, it is still capable of running a program written for the original, 16-bit 8086 microprocessor. Since then, Intel and AMD have soft-forked in both 32-bit and 64-bit instructions as well as numerous extensions like MMX, SSE, and virtualization. The Intel instruction set is grotesque. I have long been a fan of RISC instruction sets as a consequence of early exposure to programming in assembly language, and was a proud user of Linux on a DEC Alpha for nearly a decade, except that a certain US presidential candidate utterly destroyed that company.

It turns my stomach to have to accept a grotesque soft-fork and everything that comes with it. I appreciate beauty, in my equations, and clean design in my software. Bitcoin can’t have a clean design, because of the necessity to maintain continuity and not breaking things for users.

Let us come together and create a standards body to define What Is Bitcoin, in an implementation independent manner, as has been done with virtually every other critical piece of software infrastructure. The reference implementation will not survive. As with all steering committees and standards bodies, this should be composed of technical experts representing all stakeholders: businesses, coin holders, miners, and users. Most standards committees can be considered as representative democracies, because of the necessity of having experts on the committee, and the fact that not all users of software are experts. This standards body must be committed to not breaking money for everyone, as most of the core committers are. Projects that would break Bitcoin, in order to save it, are quite simply dangerous.

SolidX Partners provides consulting and strategy solutions to organizations seeking to learn about blockchain and implementation solutions. The firm also provides blockchain-based software solutions relating to the indelible recording of records, transfer of assets, and identity. For more information, please visitwww.sldx.com or email the team at info@sldx.com.

Bob McElrath

Written by

Reverse-engineering the universe and remaking money. Ph.D. Theoretical Physics; Bitcoin hacker; SolidX CTO

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade