Smart Contract Based IPFS Storage for DAOs

Jon Schwartz
Jul 24, 2019 · 7 min read

Decentralized organizations should have reliable ways to save user and application data. The storage architecture must tolerate network partitions and outages and provide control over who can save, or “pin”, data. It also needs to be easy to deploy and maintain.

One solution is an IPFS Cluster that uses Ethereum smart contracts for pinning authentication. This post goes over how this could work. It’s in the context of the Aragon ecosystem, but we’re working to make the general strategy and code useful to a larger audience.

Image for post
Image for post

IPFS Cluster

An  IPFS cluster node shows cluster endpoints connected to the cluster service, which is connected to an IPFS daemon.
An  IPFS cluster node shows cluster endpoints connected to the cluster service, which is connected to an IPFS daemon.

The IPFS Cluster Service exposes endpoints for operations on the cluster as a whole. For example, you can add a file to the underlying daemon and ask the other nodes in the cluster to add it in one call.

A group of connected IPFS Cluster nodes is an IPFS Cluster.
A group of connected IPFS Cluster nodes is an IPFS Cluster.

IPFS Cluster Controller is a client application for performing cluster operations via IPFS Cluster Service’s endpoints.

The experimental CRDT implementation of IPFS Cluster allows nodes to come and go freely from the cluster (in the current RAFT implementation, it’s recommended to have a predefined number of cluster nodes). Additionally, it enables nodes to join a cluster as “replication nodes”, which can provide extra storage for the cluster without having the ability to change the shared pinset. At IPFS Camp the record for largest cluster was set at 26.

Running a cluster without additional services satisfies our requirement for robust IPFS storage that survives network partitions and outages. If there’s a divide in the cluster network, the cluster will reach consensus on the pinset once nodes reconnect. If a single node goes down, an application can fetch data from one of the other cluster nodes. Additionally, replication nodes can donate storage space — for example, a person with an empty server at home could join a “climate change cluster” to back up important data. This is a specific use case that is important to our friends at the IPFS Consortium and AVADO.

Authentication

An authentication proxy server can intercept requests to the IPFS Cluster Service.
An authentication proxy server can intercept requests to the IPFS Cluster Service.

The proxy server intercepts requests to the IPFS Cluster Service and checks if the request to update the cluster’s shared pinset is valid. Developers are free to implement whatever authentication schemes they choose.

After surveying a few teams, we think authenticating pin requests based on Ethereum addresses is a common use case. Request Network has this need, for example. This is achievable in four steps:

  1. In the request to pin data to the cluster, the sender includes a signed message.²
  2. The proxy server hears of the request and unfurls the signature to recover the Ethereum address used to sign the message.
  3. The proxy server checks a smart contract to see if the Ethereum address used to sign the message is authorized to pin data to the cluster.
  4. If authorized, let the request pass through to the IPFS Cluster Service. If not, send back an error and do not pin the file.
The Cluster Authentication Proxy uses ethereum signatures and smart contracts to validate pin requests.
The Cluster Authentication Proxy uses ethereum signatures and smart contracts to validate pin requests.

The Aragon ecosystem has additional needs beyond authenticating Ethereum addresses, which will involve more complex solutions. For example, a DAO might want to give another smart contract permission to pin data (e.g. a token manager contract could have permission to pin), or use the outcome of an arbitrary event to determine pinning permissions. We’re still figuring out the best ways to cater to these use cases, but there are generally two strategies: using Ethereum events and provable EVM scripts. The appendix at the end of this article has an overview of these and how they might work.

Uses cases

Along with easy setup and configuration, we imagine a client-side application that shows DAO members the health and storage capacity of their cluster, and allows admins to update pinning permissions. IPFS Cluster comes with a lot of built in graphing endpoints, making it easy to display informative interfaces about your cluster. ClusterLabs provides some open source inspiration for these views.

We’re interested in understanding how we can integrate economic incentives into this architecture. For example, an organization running a cluster with extra capacity could rent excess storage to a DAO that isn’t ready to host their own infrastructure. Additionally, we’d love to explore how a Filecoin integration could be useful to DAOs.

Appendix

Event logs

One idea, taken from the IPFS Consortium, is to emit an event when you want to pin data. The emitted event should contain a hash representing data to pin. The proxy server would be listening for “Pin” events and upon hearing of one, decode the event logs, and add the cid from the event log to the cluster’s shared pinset.

Ethereum Event Logs are used to issue pin requests to the IPFS Cluster.
Ethereum Event Logs are used to issue pin requests to the IPFS Cluster.

One challenge with this strategy is handling situations where the cluster attempts to pin a cid of a file that is not yet added to any cluster nodes. This scenario would happen frequently.

For example, imagine our DAO contains a forum application that’s built on Ethereum and IPFS. We want anyone to be able to post to the forum, but we don’t want anyone to be able to freely pin data to our IPFS cluster. This is a unique situation because any user is indirectly allowed to pin their data to IPFS, since it represents a forum post and is coming from the forum application.

To achieve this, any time a new `Post` smart contract method is called, it emits a “Pin” event, which our proxy server is listening for:

Image for post
Image for post

Our proxy server hears this Pin event, gets the cid, and attempts to add this cid to the cluster’s shared pinset. But this could cause an issue — the actual file represented by this cid is not added to any cluster node yet. As a result, the cluster node has to find the associated file via the IPFS DHT. If the file is stored on a random local node that’s offline, the content may never get added to any cluster node. Future requests to fetch this content would fail.

We’re researching ways to solve this problem. One is to enhance IPFS garbage collection, so that we have more control over the timing in which blocks are removed from an IPFS node. If done right, an unauthorized user could add (not pin) a file to the cluster for a predetermined period of time. When the proxy hears of the Pin event, one of the cluster nodes will have the file, so the cluster can successfully pin it.

Evmscripts

A second method we’re researching would allow clients to construct proofs that the proxy auth server can use to determine if a request is valid or not. To reuse the previous example about an Aragon forum application, imagine if a user could communicate this:

“Hello proxy auth server! I know I’m not normally allowed to pin data to your cluster, but in this particular instance, I’m adding a new post to your forum app. Here’s my proof: [proof]!”

The proxy server knows how to verify the supplied proof, and if all checks out, will pin the data from the user to the cluster, even if this user isn’t normally allowed to.

These proofs are evmscripts. The process of “verifying a proof” is to run the evmscript in a sandbox and see if the transaction would have executed. If the transaction executed, it means the proof checks out, and the data should be pinned. If the transaction reverts, the proof is invalid and the data should not be pinned. This can all be made possible with aragon.js and Aragon forwarders. Alternatively, this proof could be computed via the Aragon ACL.

Thank Yous

[1]: Research is being conducted around dynamically updating the trusted peers array in the IPFS Cluster configuration. This feature could enable a wider range of authentication patterns but would require an additional layer of consensus within a cluster.

[2]: The message to be signed should include the CID to be pinned, along with some type of salt to ensure that any signatures captured over the wire can’t be reused to pin data (or reused to pin CIDs).

Open Work Labs

A peer-to-peer software studio.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store