Graphite V2 Architecture

If you’re interested in the decentralized web and how to manage a large application that doesn’t access user data, this post is for you. If not, this post will be incredibly boring and I recommend you skip it ;)

With the release of version 2 of Graphite last week, I built a complicated architecture designed to support multiple decentralized protocols and multiple storage options. The trick, of course, was making it seem, well, not complicated. Let’s take a look at how everything works together, but first, here’s a quick summary of what V2 of Graphite provides:

• New users can choose to sign up with Blockstack or uPort.
• uPort users can select from multiple storage provider options (Google Drive, Dropbox, IPFS).
• Blockstack users use Gaia storage, but that means they can run their own custom Gaia hubs using any storage provider they’d like.
• Users can collaborate even if they are using different authentication providers and different storage providers.

Let’s dive in!

Who’s Who

The first part of making all of this work together is knowing who is using what authentication provider and whether they have set up their storage provider yet. Now, most of this can be handled in local storage in the browser, BUT that’s only for that device and only until the user wipes the browser history and cache. So, for cross-device functionality, there needed to be a way to fetch the user’s profile, see if they had selected a storage provider or not, and if they had selected one, return the necessary information that allows the user to post and read from that storage provider.

All this means that before anything can happen in the app, I need to create a user profile with the necessary information (nothing sensitive as you’ll soon see) and store that somewhere. Blockstack does this for each user and it made sense for Graphite to do something similar. But I wanted to make this as decentralized as possible. So, I took a two-pronged approach. I would store the profile info, which you’ll see an example of below, in a MongoDB collection, but I would also store the profile info to IPFS. That way, anyone could fetch the profile information without needing to rely on Graphite or Mongo. And, even better, if the Mongo database ever went down, users could still use Graphite seamlessly because the profiles could just be fetched from IPFS.

{
"_id":{"$oid":"5c791ed1d9365b00074aa913"},
"did":"did:uport:2oqr8GRieBS2ouP22DLVnMsHY2ftXh4pge2",
"profile":{
"name":"Justin Hunter",
"did":"did:uport:2oqr8GRieBS2ouP22DLVnMsHY2ftXh4pge2"},
"storageProvider":"dropbox",
"refreshToken":"{\"iv\":\"3902d143e567497af6db9386471cb42d\",\"ephemeralPK\":\"02bf7331bf48855b9351f4fa156fd65873e4f521144c9c591a6a1f87ecf1a70033\",\"cipherText\":\"ece4d25784d214ce9dbd7711c4974306338a8994eb1a5a1868aba5c7a9585306f7158255fe03f0241e25960d3e134acecbc77b14524efde60039f79b44f04809946f70b8d2dcae5b528af7ec371eed1a\",\"mac\":\"208945dab92fe3311c1c1dcb6314455963f7af10f3a78614439f09b8d52fa71c\",\"wasString\":true}",
"publicKey":"022b8dd04bae4428ae351115b42b3ed2b1b027c98495dfdfa65b14c9abc0f6316a"
}

As you can see, there’s not much to this. It’s all publicly accessible information, and anything remotely sensitive (i.e. the refresh/access token for the storage provider) is encrypted with the user’s private key. Remember, whether it’s a Blockstack or a uPort user, every person owns their own encryption keys, so Graphite can never decrypt that data on their behalf.

One this profile is built off and saved to Mongo and IPFS, it can be accessed on each log-in. So, if a user signs out of Graphite or simply accesses Graphite from another device, the profile will be fetched, the storage token will be decrypted client-side by the user’s private key, and the user will have access to all of their data.

Collaboration

It’s all well and good that users can log in, choose a storage provider, and get to work. But what happens if that work requires them to share a file with someone? What happens if that someone is not only using a different storage provider, but a different authentication provider? And what happens if the user needs to share a file publicly?

With Blockstack, collaborating between users and sharing publicly is a solved problem. Blockstack’s Gaia storage takes this into account, and developers can easily leverage the existing tools to make this possible. But, if a uPort user is using Google Drive or Dropbox, collaboration becomes a lot trickier.

Unlike big data stores like AWS and Azure, you can’t simple make a publicly accessible folder in Dropbox or Goolge Drive. You can create share links, sure. But that’s a far cry from being able to make a simple API call to the public route for AWS or Azure and fetch data. Imagine collaborating with someone. You have your data stored in Dropbox. To share a file with this person, you’d have to programmatically create a share link, someone get that share link to the other user, then the other user would need to programmatically fetch that file and import it into the storage provider of their choice.

That’s a nightmare.

This is why I chose to leverage the power of IPFS. When a non-Blockstack user shares a file with another user, a copy of that file is encrypted (if it’s not a public file) with the public key of the recipient, and it’s stored on IPFS. The recipient simple needs the share link provided by the original user to access the file. That share link includes all the pointer info to fetch the appropriate file, and the encryption prevents unwanted parties from accessing the share link and seeing the data.

On the surface, that seems simple enough, but if you’re familiar with IPFS, storage is content-addressed. This means the identifier that points to the stored file changes every time there’s a change to the file. This is especially problematic for a Graphite Document that can change frequently.

Fortunately, IPNS and Pinata solved this problem for Graphite. IPNS can be thought of as a master record for all versions of a file. It will allow you to point to the most up to date version of a file even as the IPFS identifier (the hash) is constantly updating. Pinata, as I mentioned in a previous post makes sure the content is always available via pinning. But there’s another gotcha here.

We don’t want every old version of the document to be pinned. Those are not useful as Graphite has a built-in version control system already. So, when collaborating, I chose to make multiple API calls to ensure that only the newest version of the file is pinned. Pinata will unpin the last version and pin the new version with these API calls.

The Future

This is just a small glimpse at the new architecture. If you’d like to explore on your own, Graphite is open source and available for your inspection.

In future iterations, additional storage providers will be enabled. You can see the next up is NextCloud. There have already been many requests for storing Graphite data on NextCloud, so it seemed like the logical choice. Additionally, I want to enable Blockstack users to choose storage providers like uPort users can. Blockstack’s Gaia hub is fantastic and super powerful, but it’s largely only accessible to developers and people with deep technical knowledge.

There’s a whole world of people that can now be introduced to Graphite without fear of being locked in and unable to collaborate with others. I want to continue to expand that world and make cross-collaboration even better. I hope you’ll join me in using Graphite and helping make it better.