By Chen Li, Jims Young, Li Gong, and Ivo Entchev
1）The Dilemma of Web2
We use various apps in our daily lives: chat with friends on Whatsapp, search restaurants in review apps, explore different lifestyles on TikTok, and spend and transact money on Paypal. Our lives are captured by these apps and stored in their respective databases.
Moreover, as the complexity of our work and leisure lives grows, we tend to divide our attention across more specialized apps. For example, we could transition from a general video app to video apps that stream long, short, or medium-length content; or we would prefer tweets, long-form articles, and image-text platforms to general blogs. These increasingly refined user demands create more specific app categories but also cause user data to become fragmented across a wide array of apps.
This proliferation of specialized apps and their lack of data interoperability has led to “data silos”. We have to register accounts, create content across different apps repeatedly, and publish it multiple times to achieve “cross-platform synchronization.”
Imagine if a video we post on TikTok could be seamlessly and synchronously published on Instagram. Similarly, if a comment we leave for a video on Youtube could be synced to other platforms. When apps can share a user’s data, data is no longer trapped in silos, and users can leverage more open platforms to contribute and distribute their creative content. This is also the ultimate Web3 experience — an interoperable application matrix, as shown in Figure 1.
This trend has led to a dilemma for Web2: companies must face trade-offs between strict adherence to their business model, which relies on their monetization of trapped data, and delivering the interoperable experience demanded by their users, which requires data to be free.
Assuming that the trend towards specialized applications continues, it must be asked how Web2 platforms designed as data islands might adapt and what role Web3 will play in crafting a solution. That is the question we want to discuss here.
2) Why Web3?
Web3 world includes concepts like blockchain, cryptography, and decentralized economic models. Representative products include the cross-sovereign currency Bitcoin, decentralized finance based on smart contracts, and distributed computing and storage networks.
Taking decentralized finance as an example, the main feature of Web3 applications is the trustless management and transaction in digital assets, with user data stored in a public database, i.e., the public blockchain. There is no data isolation between applications and completely permissionless interoperability between applications on most chains (Figure 2). This degree of data freedom far exceeds that of Web2 platforms, leading to a massive explosion in decentralized finance applications. The number of applications within the EVM system grew a staggering 1,000 times between 2020 and 2022.
However, Web3 has encountered obstacles in consumer applications. The data storage capacity of public chains is minimal, and its price is extremely high (the storage cost of Ethereum is one million times that of AWS). The public chain can manage the size of data generated by decentralized financial applications but is unable to accommodate large-scale data required by typical consumer-end applications such as content and user behavior tracking data. The solution to this Web3 obstacle is to meet such storage needs off-chain and provide these resources to developers using cryptography. We will discuss the specific implementation later on.
Those off-chain storage solutions, such as Filecoin and Arweave, store data distributed on various servers in their networks by segmenting and encrypting the data. The cost is lower than centralized storage. Unlike completely open public chains, the data stored in off-chain solutions require user authorization to be called and retrieved.
Here, we need to mention a significant early concept of Web3 applications. That is, the forefront breakthrough platforms must allow users, not the platforms, to own their data, and users should also be able to hold an interest in the application platform itself. Only in this way can the platform maximize the value of data and share those benefits with its users*.
When users own data, the data isolation between applications can be bridged through user authorization. Therefore, the data center of Web3 must be user-centric. We will describe how to combine the Web3 technology layer, account management, storage, execution, etc., into a user-centric technology stack that is most convenient for developers, as conceptualized in Figure 3.
3）Web3’s Technology Stack — Starting from Defi
Compared to traditional C-end applications, Defi applications are relatively slow, and a transaction usually takes several seconds or even longer to complete. Applications based on IPFS may take at least a minute or even several hours to synchronize an update. The speed of their backend databases determines the speed of these applications.
The speed of Defi is based on the consensus of the layer1 chain, which is limited by the degree of the network’s decentralization. For non-financial applications, content generally exists on only one or a few nodes. Therefore, IPFS synchronizes addressing information, and the higher the decentralization degree of the nodes, the longer the synchronization time.
To solve the speed problem, many blockchains have developed layer 2 to store and update data that can be processed more quickly and then frequently transferred to the public blockchain. Similar things have also happened on IPFS, which is what Ceramic does. Ceramic nodes can be used, like a centralized server, to record events on one application and then update the results to IPFS after some time. It can be seen as layer 2 of IPFS.
With this layer 2, the dapp experience can be very similar to a standard app. In addition to validating dynamic data storage, Ceramic proposes data models that establish data storage standards between applications to enable cross-application data interoperability (Figure 4).
Dataverse-OS has built further resource abstraction and isolation on top of Ceramic, creating a kernel that manages storage resources and identities, similar to an operating system kernel. Applications and users can use their public keys for identity authentication to access resources without infringing on others’ data. This way, all applications can run on the same kernel rather than separate systems. By connecting to the kernel, applications can apply for permission to access any data table through the kernel, with authorization from the user’s private key, without the need to connect to each of the other applications, as shown in Figure 5.
From the perspective of data interoperability, Dataverse-OS plays the role of a cloud-based operating system, enabling large-scale data interoperability between applications. It serves as a prototype or MVP for future Web3 data middleware. There are several other projects with similar visions in this field, but we will not list them here.
4) Outlook and Challenges
Since the birth of Web3, allowing data to flow freely between applications has been one of the visions. With the development of these technology stacks, we are finally almost there.
For users, this is the first time data interoperability can be achieved across all categories of Web3 applications. We can share comments between various video platforms and discuss topics with netizens worldwide without platform restrictions. Personal assets can be used for payment on multiple platforms, and all apps will recognize some universal social credits and reputations. We can even manage our data freely, allowing it to flow between applications and users and achieve more breakthroughs in composability. For developers, data is no longer a barrier or edge to compete against other applications; only good products are.
In Web 2.0, cloud-based operating systems have become the stickiest user entry point, with more and more functions being integrated into cloud operating systems and more and more user experiences being migrated to the cloud. This is an unstoppable trend. As a result, user-centric cloud services must compete with centralized platforms for developers and users, and user-centric cloud service platforms have undeniable advantages in content and social applications.
Dataverse-OS is designed to be simple yet powerful, providing developers with all the core functionalities they need to achieve data interoperability. Therefore, we highly recommend their SDK for all dapp developers. However, we encountered a challenge faced by all infrastructure projects: they need more users, which is the primary factor attracting developers to the platform. Our answer is a project called Glitter, which focuses on building a decentralized indexing service for public data on decentralized storage. They struck gold by providing a map to the decentralized content, just like what Yahoo and Google did in the early days of the internet. We will provide a detailed introduction to this project in the following article, “Glitter: The Engine of Web3 Traffic.”
*https://onezero.medium.com/why-decentralization-matters-5e3f79f7638e
