Utreexo demonstration release
I’m excited to announce the release of the first demonstration of Utreexo. Utreexo is a new scalability technology for Bitcoin, which can make Bitcoin nodes smaller and faster while keeping the same security and privacy as full nodes.
Existing lightweight Bitcoin software often uses “SPV” or Simplified Payment Verification. While SPV allows some level of verification, it is susceptible to attacks that fully validating nodes aren’t. Utreexo fully validates, while getting some of the low-footprint benefits the SPV nodes enjoy. Utreexo replaces the “UTXO set” of Bitcoin, which is the database that keeps track of who owns which coins, with a novel cryptographic accumulator. For more info about the design, see the paper here, or this explainer by Utreexo developer kcalvinalvin here.
“Release” is somewhat arbitrary as I and others have been working on this code for a bit over a year. Shout-outs to kcalvinalvin who has been working on this since we met in Seoul last summer, as well as dergoegge and ysangkok who have come on board more recently. At this point there is still lots of work to be done, but it is in a place where tech-savvy people can download it, try it out, and see that it does some pretty interesting things.
(Note if lots of people try it out, the default server that’s set up may not be able to handle the load. You can run your own server as well.)
The current Utreexo code is able to connect to a server, download the blockchain, and verify all the transactions and signatures, and do so using hardly any disk space. Instead of a multi-gigabyte database, utreexo stores a single file of a few hundred bytes which represents the state of the blockchain. The client still needs to download the full sized blockchain (in fact, even more downloading is required) but not store it.
There is also a server component which builds the data to send the client. This takes up considerable storage space. In the current software, each client connects directly to a server to download data, but our plan is to change this into a peer-to-peer model where large numbers of nodes, some with data to serve and some without, automatically connect to each other.
The current software is a demonstration of the technology and a proof-of-concept that a wallet can work (it allows users to specify an address to import, and it gives a watch-only wallet balance) but so far does not have a true wallet and cannot make transactions to send or receive money. The software also operates on testnet, the Bitcoin testing network, and is not recommended for use with real money. There are still plenty of known bugs and inefficiencies in the code, but we’re improving it at a rapid pace.
Going forward, our plan is to modify btcd, bitcoin node software written in go, to use Utreexo instead of its current database. In doing this we can keep most of the software already written, and see how to fit it into a full node that is connected to a mix of clients, some which are using Utreexo and some which aren’t. Getting Utreexo to work with btcd may take a while, but would be exciting as it would fully show the technology operating a real peer to peer full node.
After btcd, we hope to write a specification and to try to get Utreexo capabilities into Bitcoin Core node software. While Utreexo is not a consensus change and doesn’t require a soft fork, it is a significant re-thinking of how Bitcoin works, changing consensus-critical code. It is thus likely to be difficult to get Utreexo code into Bitcoin Core, and with good reason — we want to be very sure to not introduce problems into a system handling so many people’s money. For this reason we’ve decided to try to get Utreexo capability into btcd’s code first, and in the process learn more about how it affects full node operation.
This is a first release, and there will be more coming soon. One interesting capability I hope to have working soon is the ability to verify the blockchain “backwards”, starting from the most recent and progressing back to the genesis block. While this would be extremely difficult to do with normal database-centered Bitcoin software, it’s not such a big change when using Utreexo. I look forward to writing about that new verification setup shortly. In addition, there are many performance improvements in the works.
A big reason for this release is to get people excited about this new technology — especially other open source developers. The code is up at https://github.com/mit-dci/utreexo, and a growing group of developers, from novices to experts are contributing code, comments, testing and ideas. If you’d like to join and help make Utreexo, we’d love to have you! There’s github PRs and issues, and the main communications is the IRC channel, #utreexo on freenode. We also have an audio call once a week on Tuesday mornings US time (the times change as no one time works for every time zone!)
I’m looking forward to working with lots of people around the world on this software. Bitcoin was already a global, decentralized effort, but with the pandemic, working with someone on the other side of the planet is not much different from with someone on the other side of town. This is an exciting time for Bitcoin as Utreexo is just one of the many new technologies which can take Bitcoin into its second decade of dependably unlikely 80-byte preimages.
