“Block Chains” vs “Block Pools”: A New Technology That Can Power a Decentralized Marketplace and Social Network

Is it possible to extend the concepts introduced by Bitcoin to store not only transaction data but also listings, messages, and orders to build a fully-decentralized private marketplace? In this post I cover a new type of data structure I call a “block pool” that not only allows us to achieve this, but also allows us to build other types of decentralized platforms like social networks without compromising on decentralization, censorship-resistance, or user experience compared to existing centralized alternatives.

The Drawbacks of Blockchains

Blockchains are useful for storing data that has some kind of ordering associated with it. For example, if you want to keep track of transactions, it’s important to know how they depend on each other in order to be able to compute peoples’ balances at any given time.

The downside of this is that computing everyone’s current balance requires storing the entire transaction history. In other words, you can’t “throw away” old transactions because without them you won’t know how much Bitcoin a user has. The end result is that blockchains grow steadily over time as more and more people transact on them.

While this is the only way we currently know of for storing data that has an ordering associated with it, a valid question is whether we can do better when the data we need to store is unordered. For example, if we want to store user data like the listings one has associated with their marketplace account, there’s no need to keep track of old listings when a user makes an update. Instead, we can just store the latest state of a user’s data and “forget” all the previous state they’ve uploaded. This is the key insight behind the “block pool” data structure described below that allows the Ultranet to function as the first truly decentralized private marketplace.

How a Block Pool Solves the Blockchain Bloat Problem

A block pool is a data structure I developed that is useful for storing user data in a decentralized, censorship-resistant way, and it is applicable whenever the data that one needs to store does not need to be ordered (which is often the case with user data). Conceptually, it can be thought of as a key-value store similar in schema to BigTable or Redis, with access controls determined by public/private key signatures. I provide concrete examples of how it works in the next section but the high-level principle is simple: A user registers a public key with the blockchain in a special transaction that allocates that public key some amount of space (a “block”) on all of the nodes on the network (or, if desired, on a subset of nodes on the network). When a user does this, all nodes on the network recognize that user’s public key and will allow her to store the designated amount of space on their disks. The user then has the authority to sign pieces of data with her public key, broadcast that data to the nodes on the network, and the nodes will store that data in her allocated “block.” A timestamp can be used to mark pieces of data so that nodes will replace or “forget” stale pieces of data automatically when new pieces of data come in or when enough time passes. If many users do this, the database all nodes store on behalf of users can be seen as a “pool” of blocks, hence why I call it a “block pool.” One of the key features of a block pool is that the space that a user registers is her’s and her’s alone. Thus it can be seen as introducing a novel way of solving the data ownership problem, particularly when one considers that data can be encrypted to make it visible only to select parties.

The key advantage of block pools over blockchains is that they are able to “forget” data anytime an update is made, whereas blockchains must generally store everything forever (since all future transactions generally rely on some subset of previous transactions). In that sense, the block pool concept makes updates “free” while solving the access control issues associated with maintaining an application database that were previously unsolved with Bitcoin. In the next section, I walk through a real-world example of a block pool in the Ultranet software: the listing database.

Using a Block Pool to Build a Decentralized Private Marketplace

The Ultranet is a decentralized private marketplace that uses a block pool to store merchants’ listings without compromising on censorship-resistance or user experience compared to existing alternatives. As such, in this section we provide a high-level overview of how its listing database works to serve as a concrete example of a block pool in action (the following section extends the block pool concept to building a decentralized social network). Below is how the Ultranet uses a block pool to store listings:

• First, a user registers as a merchant by committing a transaction to the blockchain. The user can burn some Ultra, the native currency of the Ultranet, when she does this in order to bootstrap her reputation on the system.
• Once a user has registered as a merchant, she gains the ability to store 20 listings with up to 5MB of data in total. Each of the twenty listing “slots” has an index associated with it in the range [0, 20).
• Note that this storage space is available only to the top 5,000 merchants in terms of reputation score, which means that nodes only need to store and index a maximum of (5,000) * (5MB) = 25GB of listing data. See section 5.4 of the Ultranet paper for more information on how reputation works.
• If a merchant’s rank drops below the 5,000-merchant threshold, their listings are dropped, but they can always re-upload them after recovering their reputation by burning more Ultra or fulfilling orders with positive reviews.
• To create a listing, the merchant simply has to create a listing message occupying a particular slot in the range [0, 20), sign it, and broadcast it to her peers.
• A node will always take the listing with the latest timestamp in a particular slot, and so if the merchant wants to update the listing later on, all she needs to do is sign a new listing message with a later timestamp and broadcast it to the network.
• Some care must be taken to rate-limit how often a merchant can broadcast an update to avoid putting too much strain on the network, but this is trivially solved by implementing a reasonable per-merchant limit on the number of updates per second (e.g. one update per second per merchant).
• More details on the system can be found in the Ultranet paper.

It is important to note that if listings were to be stored naively on the blockchain then every update would effectively require the storage of a brand new listing, making the system unsustainable. However, with the above scheme, an update to a listing deletes the old listing completely, effectively making updates to listings “free.”

Messages between users and order information can be stored using a block pool in the same way as described above.

Using a Block Pool to Build a Decentralized Social Network

In a previous blog post, I discussed how I believe there is room for cryptocurrencies to disrupt not just marketplaces, as the Ultranet is doing, but also communication platforms more broadly like Twitter and Facebook. At this point, I think it’s important to emphasize that the block pool concept seems like a critical enabling technology for this. In particular, without a block pool type of data structure, there is no way to efficiently store the data needed to run a communications platform without compromising on censorship-resistance and decentralization, thus eliminating a core value-add cryptocurrencies can provide over centralized platforms.

Below I sketch out what a decentralized Twitter might look like if implemented using a block pool:

• Users register by broadcasting a registration transaction onto the blockchain. This transaction causes nodes to allocate a small amount of space for their user data in the block pool, and associates the user’s public key with that space. Say each user gets up to 10KB of space, which is enough room for ~83 “old-school” 120-character tweets at any given time (old tweets can just be deleted once the user exceeds this limit).
• At 10KB per user, allocating 25GB of space to a block pool gives space for 2.5 million users without any scaling improvements.
• Once a user is registered, she can post tweets by signing them with her public key and broadcasting them onto the network.
• Users can “upvote” tweets by signing transactions and broadcasting them to nodes which subsequently store and index them in the block pool as well. This would eat minimally into each user’s 10KB of allocated space.
• Nodes can index and store tweets and upvotes from users and display them as a “feed.” The feed algorithm would be completely open-source and collaboratively modifiable by everyone who uses the software. Interesting custom ranking algorithms could be experimented with, like computing the PageRank of the tweet upvote graph. Users would also likely be able to choose among various open-source and private “feed ranking” options, thus increasing the diversity of posts that get viewed and cutting down on group think. Similarly, a diverse ecosystem of private and open-source spam fighting algorithms could be developed that, taken collectively, could be more robust than a system utilizing a single centralized feed algorithm.

Scaling the Block Pool Concept

While 2.5 million users seems more than sufficient to serve an initial market, in a future post I will cover strategies to scale block pools by orders of magnitude without compromising meaningfully on censorship-resistance or decentralization. For the impatient reader, these strategies can be viewed today in section 7.3 of the Ultranet paper.

Conclusion

While blockchains are useful for storing data that has some kind of ordering associated with it, they are not suitable for storing generic user data due to the fact that the entire history of a user’s data must be kept track of, which leads to unsustainable “blockchain bloat.” By leveraging the fact that most applications only require the latest version of a user’s data, block pools can solve this storage problem by strategically “forgetting” data that is no longer useful, thus avoiding the issues associated with having a constantly-growing database. This enables the development of a truly decentralized private marketplace like the Ultranet, which is launcing on February 25th, but also the development of truly decentralized censorship-resistant communication platforms. In that sense, the block pool concept can be seen as extending Bitcoin from being a database that can store only sparse transaction data to one that can store rich user data powering broader platforms that can compete with centralized alternatives.

If you liked this post, it would mean a lot to me if you could share it on Twitter.