Bitcoin Scalability: An Outside Perspective
Two weeks ago, at Scaling Bitcoin 2015 in Montreal, more than 200 people came together to discuss the future of Bitcoin infrastructure. The attendees represented an incredibly diverse group — core developers, miners, academics, policy experts, hobbyists, industry professionals, and everyone in between. The guidelines of the conference — no decisions and the Chatham House Rule — helped foster discussions and exchange ideas on how to improve Bitcoin.
It’s hard to overstate the importance of explicitly not making decisions. The Bitcoin community, many attendees admitted, stands to benefit from improved communication, both among itself and with those in other related areas of interest. Many of the problems facing Bitcoin right now have strong parallels to research in academia and industry, but those connections are only starting to be made. We think there’s a lot that the Bitcoin community can improve by building on solutions that already exist for other systems. Here’s a selection of papers and technologies we think the community should be looking at, grouped by the Bitcoin-specific problem they address.
Efficient Block Transfer
For Bitcoin to function, the blockchain has to be available for all nodes. Since Bitcoin spans the Internet, this makes data delivery an interesting challenge. During the conference, Bitcoin-specific solutions of invertible bloom lookup tables (IBLTs) and the Relay Network were presented. IBLTs reduce the data that needs to be propagated through the network on average, while the Relay Network allows blocks to take more efficient paths through the Internet and get to all nodes faster.
Meanwhile, academia and industry have produced solutions to related problems. Mosh’s State Synchronization Protocol (SSP; paper) allows two peers to efficiently reach the same state while transferring the minimal amount of data by sending only the differences. SSP uses UDP directly, and can transfer data efficiently even in the presence of packet loss. The IBLT technology provides a “logical diff” (in SSP terminology) but the current proposal is to use it over TCP. As such it doesn’t have SSP’s resilience to packet loss. Research on improving TCP for scale is also still ongoing. The Datacenter TCP (DCTCP) paper outlines a technique that could also improve overall block transfer performance.
The Relay Network is somewhat similar to Content Delivery Networks (CDNs), such as Akamai (paper). CDNs improve the availability of data through replication and geographical redundancy. Network tomography data, combined with user request load, result in mapping systems that redirect clients to the “best” location. While the Relay Network has an explicit goal of reducing latency, and not necessarily efficient bulk data delivery, many of the techniques employed by both systems utilize aspects of the physical network topology to increase efficiency. In fact, CDN techniques can be used to extend the Relay Network by using routes provisioned by commercial CDN providers, which strive for optimal topology. Akamai’s IP Application Accelerator and Google’s Cloud Load Balancing are potential building blocks.
Bootstrapping a Bitcoin node requires a huge amount of data, but most of this data is extremely static. We can increase the availability of the Bitcoin blockchain by publishing a well-established static prefix to a CDN. In this mode of operation, the CDN is performing optimally for bulk data delivery. Data transfer over long, congested links is reduced, which improves latency and throughput. A blockchain CDN, possibly built directly on a commercial CDN provider’s platform, could greatly simplify setting up new Bitcoin nodes.
Reliable (High Availability) Nodes
The Bitcoin Core codebase as implemented has the implicit intention that all participating machines would be monolithic, with comparable capabilities and running the same services. In practice, we’ve already seen a huge differentiation between the network participants: lightweight wallet devices, dedicated miners, full-node servers all exist and have different requirements. Full nodes are particularly interesting, as they perform the validation that help ensure the security of the entire network. Unfortunately, full nodes are resource intensive to run, making them fairly rare. This strongly suggests that full nodes should all be highly-available, well-sharded, and provisioned with extra capacity.
Ideally, there would just be sufficiently many full nodes that individual node availability wouldn’t be a concern, and the network could temporarily shift load if there was a significant event. Alternatively, each individual node could run across multiple machines, spreading load. Google’s Spanner database (paper) describes a highly-available and fault-tolerant methodology that also supports data sharding. Notably, Spanner uses synchronized clocks to improve the performance of the Paxos consensus algorithm. This allows Spanner to run at a global scale, similar to the Bitcoin network. For Bitcoin’s applications, that would introduce a notion of trust, which is not an acceptable design. However, an individual node can be replaced with a cluster of machines running the distributed, sharded node. This would increase the resilience of an individual node without sacrificing any of the network’s decentralized properties.
What are the limits of scale?
Although the Internet is already quite big, it still presents a scaling limit to Bitcoin and other decentralized networks. Bandwidth is improving every year, and although is expected to hit approximately 250 Tbps in 2016 (according to Cisco), most of the bandwidth is already spoken for. This makes it all the more important to keep track of the scaling horizon. While it’s not yet clear what that is for networks, Google cautions that improvements do not follow Moore’s Law while describing the Jupiter cluster fabric (paper). There’s still some room to grow; Jupiter achieves 1300 Tbps within the datacenter, and the global internet is likely to eventually reach this capacity. Nielsen’s Law seems like a more appropriate model to follow.
However, one should allow a significant margin-of-error when making projections 20 years in future. Even if 8 gigabyte blocks in 2036 sound feasible, one should account for larger network and, thus, more than 8 connections per peer to maintain low latency. Further dampening factors include those not related to network, e.g., room for cryptographic agility.
Raw bandwidth isn’t enough to make a system scale. Google further employs Bandwidth Enforcer (paper) to prevent unfair use of their network. While Bandwidth Enforcer wouldn’t be suitable on the public Internet, it could serve as a basis for curbing spam and abuse within the Bitcoin network, with the caveat that some substantial engineering effort would be needed to make it suitable for a trustless, decentralized network.
Bitcoin as a building block, not a monolith
Although Bitcoin and the surrounding ecosystem is a young and emerging technology, it’s important to remember that it extends to other communities. Just as Bitcoin provided the first decentralized append-only database supporting transactions, there are other novel systems and solutions that have interesting properties. We should work to extend Bitcoin with those advances as they mature.
This post was authored by Alex Chernyakhovsky with help from Madars Virza. Thanks to Jeremy Rubin for editing.
