A Permanent Web of Linked Data: Arweave

Photo by Omar Flores

This article is part of an on-going series on decentralized file storage for science introduced in Rich in Data, Poor in Wisdom: Science Needs a Decentralized Data Commons.

Arweave is a peer-to-peer archival protocol and network. The motivation behind Arweave is to have a place — the “permaweb” — where data can be stored permanently and cannot be manipulated. The documentation says, “Arweave is a protocol that allows you to store data permanently, sustainably, with a single upfront fee.” The platform also has a cryptocurrency, which is used to pay storage providers. Here we introduce the reader to Arweave’s actors, the novel data structure and proof behind Arweave (the blockweave and Proof of Access), Arweave’s incentive system, and the permaweb.

Actors and Basic Entities

The two types of actors in Arweave are users and miners. “Users pay tokens (AR) to add data to the network. Miners in the network receive these tokens for mining new blocks, which requires them to store and serve data” (source). A block includes metadata — such as the hash of the recall block — and transactions. Transactions, which can be created by users or miners, can store data and transfer AR. Users do not pay miners directly. Rather, the tokens spent by users are gradually distributed to miners through block rewards.

The Blockweave and Proof of Access

From the interaction of Arweave nodes emerges not a blockchain but a blockweave, which is powered by Proof of Access (PoA). In a traditional blockchain protocol, each new block is linked to the block that immediately precedes it, creating a “chain” of blocks. In Arweave, each new block includes both a link to the block that immediately precedes it and a link to some random previous block (called the recall block), creating a “weave” of blocks. Due to this use of recall blocks, Arweave dubbed the proof that powers its consensus mechanism Proof of Access. PoA is a variant of Proof of Work (PoW). In PoW, a miner must prove it has done some amount of computational work before it can append a block to the blockchain. PoA also requires computational work, but it additionally requires the miner to prove it possesses the recall block. Also unlike traditional blockchain protocols, Arweave does not require each node to store every existing block because to do so would be so impractical as to deter adoption of Arweave. The emergent incentive system both distributes the storage burden across the network and encourages miners to store rare blocks.

Incentives

Permanent Storage

Arweave’s pricing system takes into account the goal of storing data permanently. Through block rewards, Arweave miners collect transaction fees, the new tokens when the AR supply increases (though there is a supply cap of 66 million AR), and from something called the storage endowment. When a user uploads data to the Arweave network in a transaction, the total cost of the transaction includes two main components. First, it includes a one-time fee that is instantly paid to the miner who mines the block. Second, it includes a fee which is a function of the data’s size; this goes to the storage endowment. In the event that the one-time transaction fees and token supply increases of a block do not cover the miner’s costs, the miner is rewarded a portion of the storage endowment. This multifaceted approach to paying miners ensures data is stored permanently.

AIIA and Wildfire

While block rewards incentivizes the storage of data, the use of the Adaptive Interacting Incentive Agents (AIIA) game incentivizes the efficient transfer of data. The most popular agent implementation for Arweave miners is called “wildfire.” Wildfire is a ranking system in which the agent ranks its peers on two dimensions: generosity and responsiveness. A node scores higher in generosity the more it sends new transactions and blocks, lower when it sends fewer new transactions and blocks. A node scores high in responsiveness by “responding promptly to requests.” When sending new transactions or blocks, a node first sends them to nodes with the highest scores. Because every node wants to be scored highly by other nodes, the result is a system in which nodes are constantly competing with each other to quickly transfer data.

The Permaweb

Arweave’s goal is to be the center of what it calls the “permaweb.” In the vision of Arweave’s creators, we will have a permanent web of versioned documents that can be traced to their pseudonymous creators. This web will be sustained by the cryptoeconomic incentives discussed above. The benefits of the permaweb are not just permanent archival but also permissionless access and the ability to trace the development of ideas in, for example, pieces of news that might be frequently updated and that might be sources of misinformation. The permaweb would be an upgrade from the current web where censorship from large corporations or governments can be used to suppress marginalized groups, significant historical information can get lost, and tracking versions of web pages (even pages with timely news) is difficult.

Links and Resources to Arweave Ecosystem

• Arweave website. Available at https://www.arweave.org/.
• “Arweave: A Protocol for Economically Sustainable Information Permanence.” Sam Williams, Viktor Diordiiev, Lev Berman, India Raybould, and Ivan Uemlianin. Available at https://www.arweave.org/yellow-paper.pdf.
• List of Arweave applications and ways to get involved. Available at https://www.arweave.org/get-involved/community.

Conclusion

The peer-to-peer Arweave protocol creates a place to permanently archive data. Arweave’s base layer is the novel blockweave data structure and Proof of Access proof. The protocol uses these and the storage endowment to incentivize the network of storage providers to store data permanently. Good servicing behavior is incentivized with the AIIA game, specifically the wildfire implementation, wherein nodes rank each other and act nicely to increase their own rank. Arweave aspires to be the backbone of a permaweb in which all web data is archived and versioned. Arweave’s emphasis on permanent storage with a single upfront fee makes it a natural choice for use cases such as storage of scientific papers, scientific data, and news.

Join the Decentralized Open Science Movement

Does the idea of a free, open, internet of science ring a resonant chord with you? Consider joining the Opscientia community to learn, connect, and collaborate with others building a commons for co-discovery.

Articles in This Series

1. Decentralized Content Networks for a Permanent Science Data Commons: IPFS
2. Engineering Incentives for Data Storage as a Commodity: Filecoin
3. A Permanent Web of Linked Data: Arweave
4. Peer-to-Peer Storage without a Blockchain: Storj
5. One of the First Decentralized Cloud Storage Platforms: Sia
6. The World Computer’s Hard Drive: Swarm
7. A Permissionless Data Layer for Science Web Apps
8. OpSci Commons: A Decentralized and Autonomous Knowledge Commons

References

Arweave Wiki. (n.d.). Arweave. Retrieved October 24, 2021, from Arweave Wiki

Williams, S., & Diordiiev, V., & Berman, L. (n.d.). Arweave: A Protocol for Economically Sustainable Information Permanence. Retrieved December 4, 2021, from https://www.arweave.org/yellow-paper.pdf

Williams, S., & Jones, W. (2017, August 2). Archain: An Open, Irrevocable, Unforgeable and Uncensorable Archive for the Internet. Retrieved December 10, 2021, from https://www.arweave.org/whitepaper.pdf

Originally published at https://hack.opsci.io on December 13, 2021.