The Graph is now Open Source!

Published in

The Graph

4 min readAug 1, 2018

At The Graph, we’re building a decentralized protocol for querying data off blockchains and storage networks. We expect this to be a core part of the Web3 stack and to make that happen we need to foster a vibrant open community. Anyone who wants to contribute to decentralization and creating an open data economy should be able to. That’s why today we’re excited to announce The Graph is now open source!

This release is a Developer Preview that represents four months of work by our engineering team. We’re far from done, but we’ll be continuing all our development in public.

What’s in this release?

The initial implementation is a standalone indexer for Ethereum and IPFS. The node subscribes to events in Ethereum, executes developer-provided scripts to transform the data, indexes that data, and makes it available over GraphQL. The scripts run on WASM so processing the data is fast and results are deterministic. This release establishes the developer API for building on The Graph and solves many problems that teams currently have to solve themselves.

Code

There are 2 repos: graph-node and graph-cli.

Graph Node

The Graph Node is written in Rust and includes several core components:

GraphQL Server — A custom spec-compliant GraphQL implementation. We automatically generate the server from the SDL schema including support for filters w/ boolean expressions, sorting, and pagination. The GraphQL API we use is heavily inspired by Prisma.
Ethereum Adapter — Subscribes to Ethereum events using IPC or RPC and accesses smart contract storage.
WASM Runtime Host — Executes developer-provided Mappings on WASM using Parity’s excellent Wasmi library.
Postgres Store — Stores and indexes the transformed data in an EAV table (Entity, Attribute, Value) inspired by Datomic and other triple stores. The store implements filters that are used by the GraphQL resolvers. Block reorgs are handled by stored procedures that efficiently rollback and apply the new operations.

Graph CLI

Developers create projects, called subgraphs, that can be deployed to The Graph. A subgraph consists of a subgraph manifest, schema, and mappings. The subgraph manifest points to the data sources to be indexed (for example a smart contract) and any decoding information (ABI). Mappings describe how to transform or aggregate data and are written in TypeScript to make web developers feel at home. The Graph CLI compiles all this into a WASM module that can be executed by the Graph Node.

Developer Experience

We’re committed to building quality developer tools to make building on The Graph a joy. Graph CLI automatically creates TypeScript type definitions so you can access smart contract storage and event data with the help of autocomplete.

We’ve included a Getting Started Guide to walk you through the process of writing your own subgraph.

Upcoming Features

There is more work required on the core experience before we can move onto implementing the full decentralized network. Here are some upcoming features you can expect in the relatively short term: