600 Microseconds
A perspective from the Bitcoin Cash and Bitcoin Unlimited developer who discovered CVE-2018–17144
Introduction
Six hundred microseconds. That is about the time that Matt Corallo wanted to shave off of block validation with his pull request in 2016 to Bitcoin Core. 600µs is a lot less than what is saved with more efficient block propagation, like XThin, Compact Blocks, or now Graphene over typical links, especially those that are of similar low-end quality in network speed like Raspberry Pis are in compute speed. An optimization that was not in the focus by Core until XThin from Bitcoin Unlimited came onto the scene and kicked the Core team into gear on this issue. Furthermore, 600 microseconds is an order of magnitude or more below the variance between node validation speeds from a Raspberry Pi to a more high-end miner node and thus wholly in the range that the network already deals with.
This 600 microsecond optimization now resulted in CVE-2018–17144. Certainly the most catastrophic bug in recent years, and certainly one of the most catastrophic bugs in Bitcoin ever.
This bug was initially suspected to potentially cause inflation, was reported because it led to reliable crashes and confirmed by closer analysis… to be actually allowing inflation!
I have consistently and repeatedly criticized hubris and arrogance in the most prominent Core developers, and done so since 2013, when the bullshitting around the 1MB block size limit started. Here we have an optimization that talks about avoiding “duplicate” validation like validation is nothing to worry about, an afterthought in Bitcoin almost.
And a change that is quickly found to be good in peer reviewed, ACKed in Core-speak, in a rubber-stamp-like manner by Core developers such as Gregory Maxwell. Developers which I fully respect for their intelligence and knowledge by the way, but still, well, dislike as much for their overblown egos and underhanded discussion style as well as having done all they can to handicap Bitcoin with the 1MB limit.
I also have to be honest, this change creates an unavoidable element of suspicion in me. For anyone who knows what went down and what the code paths do, it is just unavoidable to have this thought here. I like to qualify that this is not what I assert nor think is happening, but definitely crosses my mind as a potentiality!
Because what is better to destroy the value of Bitcoin in the public’s eye than a silent inflation bug? What is better than creating code paths that look harmless for themselves but combined with some other, seemingly harmless rework in other areas of the code, result in utter catastrophe?
And it looks like CVE-2018–17144 would eventually have become exactly this. The only thing that saved Core is their effective client diversity between revisions and someone actually noticing that there is a problem. After two years of this bug sitting around idle and exploitable. Client diversity that has been much criticized on the Bitcoin Cash side of things, but it obviously shows its advantages now.
Reading the title of the original PR:
“Remove duplicatable duplicate-input check from CheckTransaction” ,
as well as the message therein:
“Benchmark results indicate this saves about 0.5–0.7ms during CheckBlock.”
almost reads like it could be a sick joke being played on us all now.
I always feared that someone from the bankster circles, someone injected into the Bitcoin development circles with the sole goal of wreaking unsalvageable havoc, would do exactly what happened. Injecting a silent inflation bug. Because that is what would destroy one of the very core advantages that Bitcoin has over the current status quo. That of transparency and a verifiable money supply. And, even though as a Bitcoin/BCHer, I do not see true long term prospects in Bitcoin/BTC anymore, calling the whole foundation of crypto into question just like that would have been equally disastrous to “our” variant of Bitcoin.
Now, again, I am definitely not saying this is the case with PR 9049 for sure. I actually think the explanation of a young, cocky Core developer, a new “master of the universe” wreaking havoc by sheer arrogance and hubris, is the more likely explanation.
People in general, but I don’t even exclude myself here, tend to believe in the competence of others if they appear just self-assured enough. This is part of the problem with attitude and psychological dynamics in this space. It creates a dangerous aura of ‘these guys know what they are doing’.
I myself have done some minor work on sensitive areas in the Bitcoin Unlimited implementation. And I am working on some more “consensus critical” code for BCH now (see below). And, yes, I sometimes do lose some sleep over what could go wrong. I know I make mistakes. I have done so. I will. We all do.
But I have yet to see anything resembling an admission of being imperfect by the developer in question, or any other prominent Core developer for that matter. The folks in question know exactly who I mean. There must be more reasonable folks in Core, but they are rather silent.
Much worse even: In the discussion on github that follows this PR, user freetrader (a well known anonymous but still respected member of the Bitcoin Cash community who helped to create the Bitcoin Cash initial fork) asks the very valid question:
Which is answered in the, all-too-typical for Core, smug manner by Matt Corallo, notably the original author of the bug who has all reason to be a bit more careful and respectful:
The bug was disclosed in an absolutely responsible manner. As even the full disclosure on bitcoincore.org’s own pages notices, it went to a set of trustworthy people by the person who found the bug and did so in an encrypted PGP message only.
This leaves the question why Core recklessly endangered the security of Bitcoin Cash as well endangering the myriad of altcoins that are out there and still susceptible with this premature and hasty publication. The back references from altcoins merging the change trickling into PR #14247 are a glimpse into this process.
Now, Matt talks about “running out of time” in the above reply. But what time is that exactly? If you think hard about this, this can only be a distrust in any of the informed parties that they’ll leak this secret prematurely and thus catch Bitcoin Core with their pants down, or as a worse assumption, be actually exploited by one of the informed parties against BTC.
Bitcoin Unlimited was ferociously attacked, presumably by deranged BTC supporters from the wider ‘community’, when it had a bug. And it seems a bit like Core members assumed a payback by deranged BCH supporters in kind here (I am not doubting those supporters exist), given the hints in the original disclosure that this bug has actually been discovered by someone aligned with the Bitcoin Cash side of things. But not only that, Core seems to have assumed that members on the BCH side of things who have been informed are deranged or at least irresponsible enough to leak this info to the wrong parties!
I like to applaud deadalnix and the ABC team for what I was thinking the Core team should have done here as well: Bury the fix in a bit more and unrelated refactoring code so as to fix it but also to buy some more time for an upgrade.
Maybe Core wasn’t creative enough to see a way to hide the problem, but then they also had no reason to blare it out like they did here.
This was very irresponsible, and, and this should reach any altcoin impacted by this, this is definitely solely Bitcoin Core’s responsibility.
No one else said anything in public before Core published their PR.
It should also be noted by the Core team that this creates a strong disincentive to keep them in the loop with initial disclosure for anyone finding a bug. Cory Fields has talked about the risks and dangers with regards to sitting on the knowledge of a 0-day on Bitcoin Cash, and this bug discussed herein is one that was worth at least 10x more in potential damage and thus also shorting value and angry deranged people (a.k.a. “31337 crypto trading bros”) capable of violence. If a party behaves this irresponsibly, it shouldn’t be surprised if it degrades itself to a lower position in the food chain with regards to vulnerability disclosures. I am not saying I won’t inform next time I might stumble upon something, but this is not a good way to create the necessary trust.
The Discovery and Disclosure
Sitting in my little van by the sea on Monday, I was working on getting the new CHECKDATASIG/-VERIFY opcodes that are about to activate for Bitcoin (Cash) in November implemented on the Bitcoin Unlimited client. I have been looking at a potentially neat use case for those and am motivated to get this done.
Around noon, I noticed that there is a lot of divergence in the way that signature operations counting was done in ABC vs. how it was done in Bitcoin Unlimited (BU). I agreed earlier with the BU team that I would go and port most of the CDS/-V stuff over from ABC, but I felt overwhelmed. My thoughts were that: Ok, this is doable, but this needs a lot more analysis and also many more eyeballs for review. And will take a lot longer. Sigh. While doing so, I stumbled upon this comment in the ABC code base:
// Check for duplicate inputs — note that this check is slow so we skip it in CheckBlock
My initial reaction was a slight “Eh, WTF is going on with that comment?”. And then I looked up uses of CheckRegularTransaction in ABC, which is the renamed variant of CheckTransaction in Core (but I didn’t know at that time). I dug through the code to try to understand the logic.
I noticed that block validation skips this test as it is assumed to have already happen during mempool ingress. My next thought was a bit of a sinking feeling and a “Uh-oh, I really hope the folks from ABC have thought about the difference between the mempool and block transmission and that those are distinct ways into the system. There might be a problem here!”. And then I went and thought about a way to test this. I patched an ABC node to not relay transactions even when asked and connected one unpatched and one patched node together in -regtest mode and created a transaction with a duplicate input (which the above test was skipping).
Wham! assert(), Aborted.
Next thought was along the lines: “Oh fuck, this doesn’t look good, gotta notify deadalnix and the crew what is lurking in ABC, this doesn’t look good at all. $@#%!!”. Being aware of the danger that this could maybe be further exploited towards an actual inflation and chain-splitting bug (but I didn’t further check the specifics of this, as a node crash bug with assert() failure was already enough to be worried about), I quickly and somewhat inaccurately noted to myself (and timestamped):
BitcoinABC does not check for duplicate inputs when processing a block,
only when inserting a transaction into the mempool.This is dangerous as blocks can be generated with duplicate transactions
and then sent through e.g. compact block missing transactions and avoid
hitting the mempool, creating money out of thin air./u/awemany
[Footnote: I timestamped this message in the BU slack, adding an innocuous situational lie of ‘Ooops, wrong channel’ to it. I also tried timestamping my findings on on my usual go-to site originstamp.org but they only submit timestamps every 24h due to the fees on Bitcoin being too high to do more often… I guess I should maybe get into the habit of doing timestamping transactions myself..]
Opening up a disclosure email to deadalnix, I started to have a thought of: “Ok, actually, where is this stuff coming from, when and where did they introduce it into the code, might we be lucky and this is not in a release yet?”
And then I noticed that this stuff was coming from Core. Already having written a disclosure report, I rechecked whether Core was vulnerable as well.
And, once again: Wham! assert(), Aborted.
I started to get shivers up my spine. Uh oh! Core has a crash bug, potentially worse. Stuff in the code since 2016. NOT good. NOT good at all. I like to say here that I actually had a feeling of this is bad, not this is good because of Core vs. Cash or something like that. I (unfortunately) still own a (for my poor soul significant) amount of BTC and for that reason and others do not like having bugs in Core either.
Being a responsible citizen in this space, I then wrote the encrypted disclosure email to Wladimir, sickpig and some others, attaching a variant of the ABC and the Core patch to exploit this problem to my disclosure.
I also put in a BCH address for a bounty payment to myself into that email (disclosed as proof below), as I feel this should be something worth a little performance bonus :-) No money has been received at the time of this writing yet. If you want to change this, you can send me BCH here:
bitcoincash:qr5yuq3q40u7mxwqz6xvamkfj8tg45wyus7fhqzug5
(1NBKDco2EctDXvBv6r4hqJRPWfgX9jFpqs)
I chose the handle beardnboobies as this is the first thing that came into my mind when I thought about this very discovery here. I thought: Ok, I am slowly becoming a pale nerd working on just code, with beard and manboobies. Oh well. I have noticed that this handle was — for whatever reason- taken out of the release notes that are checked into the main development branch of Bitcoin Core and is only available in the release branch / tag, being replaced with anonymous contributor on the main branch.
I wonder: Do you Core guys feel this is too unprofessional to have this pseudonym appear in the main branch? Have some humor please! :-)
By the way, a plea: I urge everyone in BCH as well as BTC (as well as impacted altcoins), to take a fine-toothed comb through the code with the goal of looking for similar issues!
More specifically, I faintly remember (though might be wrong) from discussions back with Core devs on reddit in 2016 and before, that the idea that there’s a lot of “duplicate validation” between mempool and block validation was kind of en vogue back then. Potentially more code is vulnerable because it assumes that mempool validation can stand in for block validation. I suspect more, though maybe not as grave bugs, in this area.
Reactions
After I submitted it, I felt relief and then I started to watch the space from the back. A weird situation. Only then I also fully realized what Core contributor Cory Fields described with a bit of a different angle and on a smaller scale, the weirdness of having found a bug that you know is worth millions at least, massively impacting a $100 billion currency. The fact that I could have gone and rented hash power and shorted BTC and exploited this. But also the fact that I did not!
Wladimir eventually wrote me an email that they’re preparing releases (and at that time or around it they published the PR), so I responded expressing my astonishment of the quite public handling of this serious issue.
What I was amazed by in general was the long time it took for the bug to blow up to its full proportions, with the process seemingly even not over now. One thing is certainly others digging into this and realizing the full severity of this — as it turns out, yes it CAN be used to double-spend and inflate on BTC after all! — but also the time it takes from the initial PR being public, seemingly not noticed at all and the first media article being written.
And then I noticed the usual spin. The “stupid BCashers can’t code and are irresponsible and what not” angle that is all too often repeated then by seemingly cerebrally insufficient Core supporters. I quote the below to gloat maybe. But also to show the world WHAT kind of bullshit the Bitcoin Cash side of things is facing here in a constant barrage. This is just from a few of the more prominent Core supporters and devs. There is, of course, a lot more folks foaming “btrash, bcash” at the mouth on reddit and twitter.
Tone Vays and Jimmy Song
Here we have Tone Vays, who likes to pose with the undercurrent of violence by wielding weapons on Twitter and apparently also on Youtube, discussing this bug with Jimmy Song in an unwillingly hilarious Youtube video:
Transcript of one funny part, with emphasis mine (starting approximately at 20:41):
Jimmy Song: […] It was a really big deal. And scroll up a little bit. Scroll up a little bit. Ah, F-T-Trader [sic]. So F-T-Trader is actually like a Bitcoin ABC developer. So, you know that he’s kinda trying to troll a little bit. But, you know, this affects a lot of other projects as well because they copy code from Core [laughter], so … . You know just something to think about. Deadalnix is another one.
Tone Vays [interjecting]: Another troll!
Jimmy Song: Yeah, another. Sickpig, another troll. So, Sjors is actually, like he’s a dev. I mean … Pierre has been helping out keeping track of what needs to do what. […]
Freetrader is, as said, a long member of the Bitcoin and now Bitcoin Cash community and so is sickpig. And the “troll” sickpig is actually one of the people that I have responsibly disclosed this bug to. He is Bitcoin Unlimited member like Freetrader as well and developer and he did not do any irresponsible action here, unlike Core. Their supposed trolling seems to merely consists of not agreeing with Core’s path taken.
I like to add a bit of speculation here:
What is a bit interesting about this disclosure is that this seems to elucidate a bit the structure of Bitcoin Core and the presumed “Dragon’s Den” connected to them for their supposed PR-campaigns. Their command and control structure is illuminated a bit, if you want. And one can clearly see that the inner Core circle has a list of folks that they trust more, because these folks kept quiet about the bug and the source of this bug. Likely because they indicated to them that it might come from the Bitcoin Cash side and that it might be unwise to go too far with one’s assertions, as the danger of tripping over one’s outlandish exclamations is real. I left enough hints in the initial disclosure that it came from the BCH side of things to prime them about this.
But it also looks to me like Jimmy Song and Tone Vays are not in these inner circles from Core. I suspect that it is the lack of competence that some Core members ascribe to them which keeps them from having access to more privileged information. These members should think hard why they play the role they do for the inner Core team, and might want to reassess their political alignment. That all said, I also don’t want Tone Vays suddenly praising the advantages of Bitcoin Cash on Youtube.
Luke-Jr
I like to say some words about this tweet of Luke-Jr, committing the sin of bearing false witness about us irresponsible “BCashers”…
From: https://twitter.com/LukeDashjr/status/1042977899308036096
I suspect Luke-Jr has been left in the dark about the background of this disclosure as well, not belonging to the innermost circles either. Careful observers might have noticed even more of this dynamic happening with other people.
And note again: I have done everything that is necessary to make this a responsible disclosure. The initial, unobfuscated public disclosure happened by Bitcoin Core on their github! This is exactly the opposite situation compared to what Luke-Jr is describing. This is despicable.
Closing remarks
Apart from pointing out the insane spin of some Core supporters in the preceding part, I simply want to take the opportunity now to urge caution for everyone here. Bugs lurk everywhere. Everyone is imperfect. Myself included, of course.
I started to like Jihan Wu’s credo of “Don’t play hatred, don’t wish competing coins ill. Just wish and try to make BCH better” (from twitter) and see BCH and BTC in fierce but still civil competition. Civil competition obviously meaning no violence, including no violence like attacking each other’s nodes.
I like to reiterate that, despite the gloating and strong words you might find in this article, I did everything to play fair. I also agree in general with Cory Fields from Core that it is not very easy to find the necessary disclosure addresses and information. He’s right about the lack of easily accessible GPG keys both on the BCH as well as — I like to add- on the BTC side of things. I didn’t find a non-retracted key of Pieter Wuille in time.
I also like to note that a few things went finally completely out of the window here with this bug, for example Core’s idea of ‘the code being law’. If the code is law, does that mean that you have to accept inflation now? Or is it actually the Core devs steering the ship? Is an element of reasonableness entering the space?
And yes, I sincerely believe, despite the current price ratio that BCH has a much brighter future than BTC, by being fundamentalist on the principles that matter and came along with the original white paper while not being fundamental on things that were created post-hoc — like the 1MB (now 4MW) limit in the Bitcoin Core implementation. As I also don’t think extended inflation is crucial for BTC’s operation.
But anyone is free to buy or sell as they want. Let’s continue competing. Let’s civilly inform each other of bugs. May the best chain win.
Finally, I like to thank Andrea Suisani, Andrew Stone and Peter Rizun for their review of this article and valuable input.
