The Centralized Secret

How Ethereum dApps rely on centralized IPFS hosting out of necessity

This past weekend, Matt Ober, David Mihal, and I spent 36hrs (48? idk a lot) with our favorite Ethereum devs building at ETHBerlin. Coming down from our hackathon high, this post dives into the centralized secret we discovered and how we built Pinata Overlay to solve it.

The Centralized Secret

Typical IPFS storage is currently centrally built and managed by the dApp company in their dApp stack.

As you look around at dApps in the Ethereum ecosystem, it becomes apparent that companies hate the current state of IPFS infrastructure. As a concept, IPFS is a powerful protocol that companies use to complement their dApps by providing decentralized content storage. However, if you look under the hood, you will find most dApp companies are centralizing IPFS’s decentralized protocol because of a lack of time and resources. In practice, these dApp companies don’t have the dev time, the monetary resources, or the tools to properly decentralize their IPFS infrastructure. And, to be honest, it’s unfair to expect dApps to have to design token mechanics, solve their user-base’s problems and waste their valuable time building their own decentralized IPFS infrastructure for the sake of decentralization.

Ethereum ecosystem best practices using IPFS are a huge centralization weakness that incurs major costs in terms of time and resources from dev teams.

So what do dApps do? Most dApps simply host an extremely centralized IPFS node on something like AWS. Or, they might run it through INFURA (no shade…we love INFURA). Either way, both of these methods end up with the dApp having a centralized failure point.

Pinata Overlay Solves This By…

The Overlay smart contract funds and pins content while the IPFS oracle allows IPFS node providers to find files to pin.

Pinata Overlay solves this failure point through a smart contract and IPFS oracle that allows for decentralized funding of IPFS storage. Our ecosystem allows dApp companies to utilize decentralized IPFS nodes without the need to build and host their own IPFS infrastructure. This solution provides the necessary IPFS infrastructure that dApp companies desire while decentralizing the IPFS hosting itself. This allows dApps to focus on the thing that matters most: their users.

The Pinata Overlay team deep in the night at ETHBerlin

More Technical Details

Pinata Overlay consists of two main features:

1. A smart contract where anybody in the ecosystem can register another smart contract to pay for all of its IPFS hashes to be stored by a decentralized network of hosting providers.
2. A decentralized network of hosting providers running Pinata Overlay’s open source node software. These providers are constantly watching the Pinata Overlay smart contract for registrations. When a provider is registered and funded to pin a specific smart contract, it automatically pulls a configuration file that was created for the funded contract from IPFS. It then pins all tagged IPFS content for the funded contract.

These features sufficiently provide decentralized IPFS infrastructure to any dApp that needs it by giving them the option of different IPFS pinning services and providers that best suite a dApp’s requirements.

Pinata Overlay Demo

Pinata Overlay Interface

To use Pinata Overlay, the first step is to upload the ABI information from your smart contract. This can easily be found on the contract’s code tab of Etherscan as shown below:

Smart Contract ABI

Once you have copied the ABI from Etherscan, you will click the “Import” button. Once clicked, paste the ABI into the window. Then, click “Submit ABI Code”:

Submit ABI Code Window

You will now be shown a list of your smart contract’s events and values in those events. Simply select which event values have IPFS hashes and need to be pinned. Once finished, you will confirm your choices and a configuration file will be generated and uploaded to IPFS for hosts to read:

Select event values that have IPFS hashes

When you have your event values confirmed, you can then chose which host or hosts that you would like to use for pinning that content. Hosts can be in the Pinata Overlay network by downloading and running our open-source IPFS client. Or, hosts can also develop their own custom solution. Once in the network, these hosts have different statistics such as uptime, location, cost, etc. Here, we’ve chosen to use our centralized IPFS solution, pinata.cloud, to host our files. If you wish for more decentralization or redundancy you can choose more hosts to pin content:

Select the IPFS host

You will then paste the smart contract address associated with those IPFS files and click “Register” to register the contract with your selected hosts:

Registering the smart contract address

Once clicked, you will be prompted with MetaMask to pay for registration:

Contract registration (sorry, I was out of test ETH)

To fund the pinning of the content, you top off the balance of the contract for the time you want to pin content based on your chosen host’s costs.

Onstage presenting after winning at ETHBerlin!

And, yay! You now have decentralized IPFS infrastructure! If you are interested in Pinata Overlay, please reach out to us at team@pinata.cloud. You can also find more details on our devpost submission page. Thanks to all the people and companies at ETHBerlin who verified this centralized secret and helped us build.