Introducing GraphGen: A subgraph generator for The Graph Network

This blogpost introduces GraphGen: a command line tool used to generate subgraphs for The Graph Network using nothing but solidity interface files annotated with special GraphGen commands. GraphGen would not exist today without the generous support of The Graph Foundation and their grants program 🍾.

When The Graph Network was announced in mid 2018, it was first and foremost a vision of a decentralized web3 stack, from consensus layer to web interface. Today, 158 indexers are indexing data for dozens of protocols across 205 subgraphs. As The Graph Network has become an increasingly critical piece of the web3 stack, so have subgraphs become a necessity for protocol developers making their forays into this exciting new internet.

💡From frustration to automation

Although far from being an overly complicated process, writing a subgraph for a given protocol is nonetheless not the easiest task; creating the simplest subgraph still requires a good understanding of GraphQL, Typescript and how The Graph works internally. This can put additional stress on protocol developers, the latter often organized in small teams already overstretched by the fast moving industry. Moreover, no standard subgraph writing practices have so far gained mainstream adoption among subgraph developers (although the good folks at OpenZeppelin have made some progress on this front), which further complicates the process for teams trying to make a subgraph for their first dApp.

These two problems (lack of solutions for simple subgraphs and lack of standardization) are what motivated the development of GraphGen. What started as an internal tool to generate subgraph skeletons was improved to the point where one can generate an entire subgraph (i.e.: manifest, schema, mappings…) from an annotated solidity interface file!

Most subgraphs deployed today remain relatively simple in terms of data transformations and indexing. The same patterns emerge in almost all subgraphs:

• Start from a “factory” type contract
• Use events or function calls to find new contracts
• Store certain events
• Update certain fields
• …

Moreover, in most cases, there exists a certain symmetry between the contracts and events stored on the blockchain and the entities stored in a subgraph. In the most extreme of cases, subgraphs essentially become a 1-to-1 mapping between contracts, their events and the entities representing the latter (i.e.: the same fields being present in both). This meant that subgraph writing lends itself extremely well to automation.

🔮 How it works

At a high level, GraphGen works by parsing solidity interface files annotated with special comments and using the definitions of the interfaces along with the annotations to generate a subgraph. Below is an example of a solidity file containing interfaces with GraphGen annotations.

Uniswap V2 interfaces with GraphGen annotations

This is enough to generate a complete working subgraph that indexes the Uniswap V2 Factory and Pair contracts, as well as the pair contracts’ Mint, Burn and Swap events, while also updating the price0 and price1. A complete breakdown of this example can be found on our github repository .

🎶 Annotations

As can be seem with the example above, the GraphGen annotations are designed to be self-explanatory and lightweight. Only three types of annotations are currently supported by GraphGen:

• @gg:source: Defines a dataSource or template (depending on parameters). Must precede an interface definition.
• @gg:field: Defines an entity field with an optional default value. Must precede a view or pure function definition.
• @gg:handler: Defines an event (or call) handler, as well as the logic to be executed when the handler is triggered. Must precede an event (or call) definition.

❗️Handler actions

Along with these three annotations, GraphGen also supports four actions for @gg:handlerannotations:

• StoreEvent: When the handler is associated with an event, this action command tells GraphGen to: 1) Create a new entity type for the event; 2) Store each of those events; and 3) Add edges between the emitter contract entity and the event entities.
• StoreCall: Same as StoreEvent, but for storing function calls. (Note: StoreEvent and StoreCall might get merged into a single action command in the future).
• UpdateField X: This action command tells GraphGen that whenever the handler is triggered, the field X (of the emitter contract entity) should be updated. The field X must be defined using a @gg:field annotation on the same interface.
• NewEntity X of Y: This action command tells GraphGen that whenever the handler is triggered, a new template contract entity of type X should be created, using the field Y of the event (or call) as the address and id of this new entity. X must be defined using a @gg:source annotation.

We refer to the repo’s README for a thorough description of GraphGen annotations.

These annotations, although not powerful enough to generate every possible subgraph, are nonetheless powerful enough to generate a basic working subgraph. The generated subgraph can always be edited manually if need be.

Important caveat: GraphGen is still in active development and should only be used in production with caution. Moreover, the generated subgraphs might not always be optimal from an indexing point of view.

Going forward

This is only the first version of GraphGen and we look forward to The Graph community’s feedback. In fact, we invite anyone in The Graph community to become co-creators in this project. For any feature request or bug report, we recommend creating an issue in GraphGen’s repository here.

🙋🏽About us @Protean labs

We are a small team of recent engineering graduates and crypto natives based in Montreal, Canada. Each team member shares a deep passion for decentralization. We envision a future where the internet’s economy and its incentives are better aligned with the broader society.

To pursue these ideals and organize our common projects we created Protean Labs, a company with the current goal of building various pieces of infrastructure and developer tooling around the Web3 ecosystem. GraphGen is our first open-source public-facing project which is made possible by The Graph Foundation and their grants program. We are thrilled to have received grants for 2 different projects, GraphGen and The Mesh Engine & Language. More details coming soon for the latter…

Stay tuned for upcoming (soon 😏) announcements on our next endeavors!

Christophe, Thierry & Mathieu