An XT FAQ

It’s been a little over 10 days since the announcement of Bitcoin XT with BIP 101 support. The XT website has an FAQ but some questions keep popping up over and over, so I want to address them here too.

Does XT make Mike and Gavin dictators?

Some people are portraying XT vs Core as “Dictatorship vs Meritocratic Consensus”. This isn’t right.

Gavin and I can’t be ‘dictators’ because all we do is write software: if we go crazy, or do other things you disagree with, XT can be forked in exactly the same way as we did.

But that’s not the real reason that statement is wrong. Many don’t seem to realise this, but Bitcoin Core uses exactly the same leadership model as XT for everything except the block size debate. Virtually all code changes in Core are not made by consensus, but rather, by Wladimir van der Laan (the maintainer) making a decision by himself.

We can see this easily in a disagreement that flared up on Monday about whether support for serving lightweight/P2P smartphone wallets should be made optional:

But no, this won’t convince me that this change is not “needed” ….. Full community consensus is important for changes to the consensus code, certainly for hard forks. Not for an optional behavior change in the P2P network code.
— laanwj

Oh look, Wladimir ignoring lack of consensus and being a dictator! Except …. no. He’s just being an ordinary maintainer, and doing what maintainers are meant to do: make decisions. Which is fine, and as it should be. Several developers including me and Gavin happen to disagree with his decision, and we’ll get to that later, but disagreements within a group of people is nothing special.

The details of that particular dispute are obscure and not that interesting; my goal here is to talk about decision making.

The differences between Bitcoin XT and Bitcoin Core’s decision making process are actually very small, and boil down to this:

Do maintainers make decisions for all code changes submitted, or all of them except those related to a specific type of protocol change?

It’s worth remembering that for most of Bitcoin Core’s history the answer was the first: the maintainer made decisions for all code changes. This continued up until Gavin stepped back to focus on research, which wasn’t as long ago as it feels: it was in April 2014. The idea that unanimity is required for rule changes to get into Core is actually very new.

An example from a few years ago will make this clearer.

The story of P2SH

In 2012 the Bitcoin community realised that it needed to make support for multi-signature wallets easier to use. Satoshi had put support for multi-sig in the protocol right from day one, but the problem was it was awkward to use because it didn’t work with Bitcoin addresses, and addresses were (and still are) a really popular way to ask for payments.

After some debate, the community decided to create a new kind of Bitcoin address that would fix this. To do it would require some new features in the Bitcoin protocol and block chain. It would require …. changing the rules.

You can guess what happened next.

Yup, there was a massive drama and a disagreement that was put to a miner vote. One proposal was called P2SH and it’s what we use today. The other was called OP_EVAL.

This post by edmundedgar on reddit gives a good summary:

The P2SH rollout was a huge, slow-moving psycho-drama with people putting NoP2SH in their blocks. If we’d had to wait for a full developer consensus (not to mention the kind of world+dog wider-world consensus some of the core devs are saying is needed here) we’d probably still be waiting for it to get done.

The details of that dispute were so arcane that I actually can’t remember what they were. The proposal that won is described in BIP 16. I guess it seemed like a big deal at the time.

The P2SH soap opera had all the same features the block size has:

• Miners voting for what they wanted by putting markers into their blocks
• A mini-fork of Bitcoin. It didn’t have a logo, it was just called the OP_EVAL patches and you had to compile it yourself, but it was there.
• A voting threshold of 55% (much lower than the 75% that BIP 101 uses!)
• Giant and seemingly never-ending debates splatted all over reddit, bitcointalk etc. Developers got testy with each other.

Here are some things it didn’t have:

• Rampant censorship in the community forums
• Accusations of destroying Bitcoin, etc
• Companies signing letters to indicate their support
• Media stories about a Bitcoin civil war

You may wonder why the P2SH rollout was so much less traumatic.

If you ask your local Bitcoin Core developer, they might mumble something about soft vs hard forks, but that’s a distraction: the differences between the two types are tiny and cease to matter entirely once the rollout is over and everyone has upgraded.

The real reason for the difference is that back then Gavin was the maintainer of Bitcoin Core and he wasn’t afraid to pick something and require a simple majority to activate it, whereas now Wladimir is the maintainer and he requires unanimity. It seems literally every last person in the Bitcoin universe must agree or else a change cannot happen. We think, based on experience, that this can’t ever work.

So in the end, this is all XT is doing leadership wise: going back to the exact same model of decision making we used up until April 2014.

(as an aside, I was one of the people who disagreed with Gavin about P2SH. I favoured a different, more ambitious approach. But Gavin thought the community wasn’t ready for it; hindsight proved him right and me wrong)

Why is XT more than just BIP 101?

This is actually answered on the website already, but it’s one of the most common questions so I’ll answer it again here.

tl;dr you can get Core+BIP101 here if you want it.

Now why didn’t we provide that ourselves?

It’s not well known, but Bitcoin XT predates the current block size debate. When Gavin realised Core was deadlocked, he decided he’d submit his work to XT because he knew I’d already created it last year. Not many people ran it until now and it didn’t even have a logo or website, but it was there.

XT was created due to frustrations with the way Bitcoin Core became after Gavin left, of which the block size debate is only one manifestation:

• An inability to recognise that different people can be experts and yet reach different conclusions from the same facts.
• An extreme level of black/white thinking, in which something is either mathematically perfect or hopelessly flawed to the extent it shouldn’t exist at all, with nothing in between. If the mathematically perfect thing doesn’t actually exist yet or nobody knows how to build it, that doesn’t matter. Given the dominance of people who love mathematics in the current dev team, this approach isn’t totally surprising.
• Outrage based decision making (or non decision making).
• An exceptionally aggressive and toxic environment in which people’s work is routinely equated with their personality: as just one example, arguing that something is “good enough for now” is often seen as a character flaw or evidence of idiocy, rather than merely having differing priorities or experience. As another example, if someone actually writes some code and it turns out to contain a single bug, even a bug missed by others during peer review, that’s presented as evidence of carelessness or some other personal failing.
• Changes that appear to be technical no-brainers are frequently shot down for unpredictable reasons, which often don’t make much sense.

For many of us, especially those of us with experience working at well run software engineering firms, these sorts of experiences have made working on Bitcoin miserable …. where once upon a time it was fun.

To quote one developer who got sick of Core:

I have some patches in bitcoin core and stopped contributing because of the core developers having discussions that were really not technical or at all
predictable. They seemed just personal preferences which made it hard to
predict if something I did was going to be included or not.

After this fork I see that the discussions are much more healthy and technical in nature, as such I’m probably going to contribute again after my holidays.

Bitcoin XT exists partly to create a fresh environment with a different approach. There are more people out there like that, and some have already shown up. The manifesto on our website describes some concrete ways we’re doing this.

One of the most important differences is we recognise that things in life are rarely perfect, and often it’s alright to start out as “good enough for now” and then improve in future versions. A great example is unconfirmed transactions.

Even before BIP 101 happened, XT included work by Gavin Andresen and Tom Harding which makes it easier to spot double spending, thus helping merchants protect themselves against fraud. After some debate and improvements, Gavin merged it into Bitcoin Core.

Once the feature was added to the code, a bug was found by the additional exposure. This happened before the next release, so only people deliberately testing the ‘bleeding edge’ code that wasn’t yet formally announced would have encountered the issue, which was rare and not spotted during code review.

Instead of fixing the bug and continuing, the feature was removed (not by Gavin). The criteria for being re-added was said to be:

1. More testing (but how much and how to get it was left unspecified)
2. Stop being controversial

Unfortunately the reason the feature was controversial was logic like this: It’s not clear if the feature is perfect. If it isn’t perfect, it would be snake-oil medicine and it’s better for it to not exist at all.

This line of thinking is frequently paired with “Bitcoin has a multi-billion dollar market cap. If you support a change that’s potentially imperfect you are being reckless with other people’s money”.

Needless to say, this makes it hard to get anything done: increasing the block size is stuck in exactly the same limbo. I’m sure some of the Bitcoin Core guys would argue with this depiction, nonetheless, it’s the perception of multiple developers, not just me.

Some say that if XT had less stuff in it, that’d make it easier to increase the block size. I don’t think it’d make any difference given that miners can easily run custom software, but anyway, if we don’t fix the underlying issues the same thing will just happen again in future.

Did BIP101 bypass peer review?

No.

That claim cropped up in this IEEE Spectrum article:

In a way it’s less the details of the proposal and more the attempt to bypass the scientific review process and impose one design by lobbying, blogging, and populist campaign

It’s unclear what “scientific review process” refers to here: later in the article the NIST SHA competitions are mentioned, but those have an organising committee that has a clear process for making a final decision. Bitcoin has nothing like that.

Suffice it to say BIP 101 was submitted to the Bitcoin Core peer review process here and here. Note that very few useful comments were received: the code review got cosmetic suggestions (code should go in this file instead of that file etc) and a comment from Peter Todd saying he and others were going to ignore the review request because it had no chance of being accepted.

In fact there were bugs. They were found and fixed when BIP 101 was submitted to the Bitcoin XT review process. Far from bypassing the review process, it went through two of them, but only the second actually got results.

So if you’re concerned because you think Bitcoin XT means an end to peer review, don’t worry.

Does the privacy-hating XT blacklist Tor?

No, Tor still works just fine with XT.

The origin of this meme is that during the XT testing cycle we discovered that someone was trying to do denial of service attacks specifically on XT nodes (i.e. flooding them with traffic to try and jam them), presumably in an attempt to scare people away from the project. They were doing this through Tor.

Bitcoin Core has a very black/white view of DoS attacks. Either you’re banned or you’re not banned. Obviously, banning Tor is an easy solution to such attacks but is a last resort: a few legit users do connect through Tor as well.

So Bitcoin XT has been upgraded to have a more nuanced view: connections get a priority score. Then if your node runs out of resources i.e. is under attack, it chooses to serve high priority connections first and low priority last.

Put more simply: when there’s no attacks happening through Tor, nothing changes. If there is an attack happening through Tor, the legit users on it get served with same importance as the DoS attackers on it — last.

This logic reflects a cold reality: proxy networks like Tor are slightly appealing to ordinary users but very appealing to attackers. I used to work with Google’s anti-DoS/security systems closely and so am quite familiar with handling this kind of abuse. Our approach ensures Tor is still usable whilst mitigating risks to the network.

For “Tor” in the above explanation you can read “Tor+whatever other open proxy networks we see being abused”. The logic should be extended in future.

This work was submitted to Bitcoin Core. They accepted that the idea of IP prioritisation makes sense, but were appalled by the idea of having an actual list of IP addresses that are being used by attackers, because it’d be “centralising”. Luckily the makers of popular email servers don’t think the same way, otherwise spam filters wouldn’t exist.

XT takes a simpler approach. Fighting DoS attacks is hard. You start by grabbing the lowest hanging fruit. Over time you can evolve the code to be smarter, more automatic and to learn things by itself instead of having to be told by people. But that’s months of work, and you have to start somewhere.

Be pragmatic, start somewhere practical, value code that exists, iterate towards something better: this is the XT way.

I hope this article helps clear up some misunderstandings about what we’re doing. Thanks for reading.