Storage That Never Dies
LiquidStorage, the Incentivized IPFS for Files & Websites
TL;DR
- Storage systems, the backbone of the internet, are struggling to keep up with technology and exponential growing data.
- The InterPlanetary File System (IPFS) was built to disrupt centralized file storage, spreading data across a distributed network.
- Yet IPFS nodes have no incentive to keep files readily available, making it impossible to guarantee quality service. Protocol Labs, the creators of IPFS, began the extensive Filecoin project to address this issue.
- The free market for DAPP Service Providers (DSPs) already represents an optimal incentive layer for IPFS. The DAPP Network has proven synergistic with IPFS, with its staking-based free market incentivizing first-rate service. Combined with the DAPP Network, IPFS can finally upend traditional storage models and become the backbone for Web3.0.
- Want to get started? Take the first steps to hosting your websites or files on LiquidStorage by setting up with Zeus SDK or Zeus IDE, and reach out to us on Telegram with any questions.
The Promise and Pains of IPFS
Many teams across the globe are hard at work decentralizing the internet in order to realign it with the equitable spirit in which it was founded.
However, the storage systems underpinning the centralized web are struggling to keep pace with both the growing volume of data flow, and the global shift towards decentralization.
Recognizing the limitations of these web storage system, the team at Protocol Labs designed the InterPlanetary File System with the aim of upending the traditional client-server model of online web storage. Instead of storing data in centralized servers, IPFS distributes multiple copies of a single file across a decentralized network of nodes, making it a crucial piece of infrastructure for the decentralized web.
Yet IPFS has an Achilles heel preventing it from scaling to service millions of websites and applications worldwide. (Click to Tweet)
Nodes on the IPFS network have no incentive to keep files available for quick retrieval, making it impossible to guarantee quality service on IPFS. Without a robust incentive layer to complement IPFS, user experience on the network will not be able to match that of the traditional systems it seeks to replace.
While Protocol Labs launched the extensive Filecoin project to rectify this very issue, the free market for DAPP Service Providers (DSPs) represents an optimal incentive layer for IPFS.
IPFS has already proven to be incredibly synergistic with the DAPP Network. It forms an integral component of the vRAM system running on the network, which is providing production-grade memory for dApp developers. DAPP Service Providers (DSPs) serve as the interface between IPFS and the blockchain by hosting both high-performance blockchain nodes and a local IPFS cluster. Developers access DSP services by staking DAPP tokens to their preferred service packages, which are offered by DSPs on a free market.
Together with an incentive layer based on the free market for DSPs, IPFS can finally reach its lofty goal of underpinning the decentralized web.
IPFS and Its Data Availability Problem
A decentralized web requires a means of storing files in a distributed, trustless fashion.
Launched in early 2015, the InterPlanetary File System (IPFS) has set out to transform the way we send and receive files on the internet. IPFS seeks to replace the old model where files are stored on centralized servers and transferred to requesting clients via HTTP.
While immensely successful for the first iteration of the Web, the client-server model seems to be threatened by the evolution of technology and growth of data:
- Host servers centralize the data, introducing a single point of failure into the system. Failure to deliver a requested file could result from censorship, the server being offline or a large number of requests incapacitating the server’s ability to supply the requested file.
- As data is generated in ever-growing sizes, enhanced storage methods that are both censorship-resistance and enable efficient data access must be implemented in order to ensure a smooth user experience.
IPFS intends on becoming the backbone for Web3.0.
It disrupts the traditional centralized file storage model by spreading information across a distributed network, allowing the data to become accessible from multiple nodes. Traditional file storage systems utilize location-based addressing in order to locate a specific file. A client requests a file from a server by pointing to its IP address, which the server than uses to locate the file.
With IPFS, the order is inverted.
Each file is given a Unique Resource Identifier which is also the hash of the file. Clients requesting files in content-based addressing systems such as IPFS first point to the identifier associated with the piece of content, after which the system can then determine the physical location of the data and retrieve it.
In addition to allowing clients to fetch the nearest possible copy of the file thus reducing latency, an IPFS URI also ensures the integrity of the data. Since each URI is also a cryptographic hash of the file itself, data stored on the IPFS network cannot be altered without changing the data identifier. When a user uploads a new file, it is first hosted on their local machines before being propagated to requesting nodes on a peer-to-peer network. On the DAPP Network, file URIs are stored on-chain, making it simple for the data’s owner to verify the integrity of their information.
The technology is inspired by previously existing peer-to-peer systems such as BitTorrent, an exchange protocol used to share large data files in a distributed manner, and Distributed Hash Tables (DHT), a key-value lookup service for retrieving stored values in distributed systems.
On its own, however, IPFS is not a complete solution. For a decentralized storage network to succeed in supplanting the client-server model, IPFS must be fused with an incentive mechanism that holds nodes accountable to high standards of service delivery.
IPFS nodes store what they deem is important, so there is no guarantee that the node storing your file will keep the data available for efficient retrieval. Nodes may choose to remove your file to make room for higher-priority data. Furthermore, there are no service guarantees ensuring that an IPFS node remains online in order to deliver any requested files with minimum latency. This introduces unnecessary points of friction that severely hamper adoption. If you wish to store your website frontend on IPFS, for example, you cannot afford to risk users having to wait for the site to load.
Data availability issues create a poor user experience on IPFS.
Without a way of incentivizing nodes to provide a high-quality service, it is unlikely that this new system will serve as the foundational component of Web3.0.
Protocol Labs understood the limitations of IPFS and, for this reason, set out to build Filecoin as a complementary component forming a comprehensive decentralized storage network. While IPFS nodes may fail to deliver requested files either due to being offline or because the file no longer exists on them, users can make use of a ‘pinning’ function which allows them to upload data onto a node that will remain online and “ordering” that node to keep the data.
https://docs.ipfs.io/guides/concepts/pinning/
Under the Filecoin system’s design, users can go about utilizing the ‘pinning function’ in two ways: either they can manually enter into an agreement with a given storage provider or they can purchase storage capacity on Filecoin’s algorithmic market. Individuals and organizations with unused storage space can add it to the Filecoin Network and receive tokenized rewards in exchange for storing data on their nodes.
Filecoin presents one alternative to solving IPFS’ data availability problem with a decentralized marketplace for data storage. But it’s not the only one.
Synergies Abound Between IPFS and the DAPP Network
As mentioned above, IPFS plays an integral role in the DAPP Network’s tech stack.
IPFS already plays an integral role in the DAPP Network’s tech stack. This development takes that one step further, to storage that never dies. (Click to Tweet)
The first service to launch on the DAPP Network, the vRAM System, uses IPFS behind the scenes to make data readily accessible to EOS dApps. vRAM is an alternative memory solution for developers building EOS dApps that is RAM-compatible, decentralized, and enables storing and retrieving of potentially unlimited amounts of data, affordably and efficiently. vRAM itself (and thus IPFS) is used by other DAPP Network services such as LiquidAccounts, as well.
EOS dApps used to be constrained by the limited scope of RAM, an EOS-specific storage resource for housing smart contracts and their relevant data. Unlike its name implies, EOS RAM does not serve as cache layer, but rather is used to store all smart contract information in a limited database.
With vRAM, dApp developers can store their data on chain history instead of directly in RAM. DAPP Service Providers (DSPs) index the data and maintain a copy of the file on their IPFS node, harnessing LiquidApps’ patented technology to deliver the data securely and efficiently. In order to access the vRAM system, developers stake the required amount of DAPP tokens to their chosen DSP’s service package or lease DSP quota from a community-provided leasing market.
The synergies between IPFS and the DAPP Network do not end there.
The staking-based free market for DSPs which sits at the heart of the DAPP Network is designed to incentivize first-rate service delivery, making it a prime candidate to fulfill the functions of an incentive layer for a decentralized storage network.
Users are free to choose which DSP to use based on their price, service level agreements (SLA) and reputation for service delivery, creating a natural incentive for DSPs to remain honest and reliable. Services are procured through staking, giving users have an easy way to hold DSPs accountable for bad behaviour. Users stake DAPP tokens to DSP service packages in order to access storage services, and the DSP receives its reward in the form of a proportional share of the annual inflation of DAPP tokens. DSPs can then “pin” files on behalf of their users, delivering them upon request. Should they be dissatisfied with a DSP’s performance — which is permanently auditable on chain — users can simply unstake their tokens, penalizing the DSP’s future earnings in the process.
LiquidStorage: An Incentivized, High-Performance Storage System
The developers of IPFS had big dreams for their creation. It was envisioned to become the backbone of Web3.0 — a decentralized storage system that could handle the explosion in the volume, variety and velocity of data generated online, replacing the traditional client-server model. Yet IPFS never stood a chance on its own. Protocol Labs conceived of Filecoin as IPFS’ essential twin. Filecoin was designed to plug a gaping hole in the system by incentivizing providers to store and deliver files quickly and honestly.
Yet, 2 years on from their $257 million ICO, Filecoin’s marketplace only recently entered the testnet phase, while IPFS continues to search for the missing piece to solve its data availability issues. Meanwhile, the natural synergies between the decentralized storage network and blockchain technology are already on live display on the DAPP Network. A DSP-based marketplace for decentralized storage and its unique market dynamics make it the optimal way of incentivizing honest, high-performance service delivery on IPFS. With a storage system that can scale, the decentralized web will finally be well-poised to deliver on the promise of an equitable internet with enhanced data protection for all.
