Complex Blockchain Queries using The Graph
A Story of CryptoKitties and GraphQL
At Rockside, we are constantly doing a technology watch of the new Ethereum products and see how we could fulfill promises that meet bread-and-butter issues for Ethereum developers.
In the team we agreed on one particular recurring issue in the Blockchain world: querying data is a pain! By queries, we mean something rather complex that would require several (if not a ton) of requests to your node in order to get the desired result. Here is our feedback experience.
After reviewing several options, we stopped at a solution that seemed to stand out from the others: The Graph, “a decentralized protocol for indexing and querying data from blockchains”.
The CryptoKitty Challenge
In their documentation, The Graph highlights an example of a complex query involving the famous collectible CryptoKitties that asks the following question: ‘Who are the owners of the CryptoKitties born between January and February of 2018?’
The team states that this kind of request, otherwise very painful to implement, could be undertaken really easily through The Graph.
Problems encountered when done Manually
As we like challenges at Rockside, we tried to answer that question first without any other tool than one of our ethereum node and web3js. During that process we encountered three main issues:
- Ethereum does not have an easy way to query blocks by timestamp
- There is huge load of data to filter and decode(~340k blocks for a one month period)
- For this kind of query we are very dependent on the methods that the smart contract exposes publicly, and it can potentially take ages to gather the data needed
If you are curious about the script we came up with, you can find it here
Solving the query using The Graph
Structure of the software
The Graph is composed of several parts:
- the Graph-node that scans your node and indexes data
- Subgraphs that give instructions on which data should be indexed from the Blockchain.
- a WASM module attached to the Subgraph that will map ethereum data into GraphQL schemas
- a Store where the data is persisted with PostgreSQL
Making our own Subgraph for indexing Cryptokitties
The installation process is fairly smooth, you are just asked to select a network, the Ethereum address of a smart contract to watch and it will fetch the ABI directly from Etherscan if possible. Your Subgraph is ready to index the main events fired by the smart contract in less than 2 minutes.
Now, the Subgraph as is will not be enough to answer our question. It needs some extra elements in order to be able to query Cryptokitties and their current owner in one go.
Customizing a subgraph can be done a few simple steps:
- Define an entity that The Graph will persist in its store in the Subgraph manifest
- Define the GraphQL schema of that entity in the GraphQL schema file
- Define the corresponding handlers that the graph will react to in the Subgraph manifest
- Customize the handlers to map the ethereum data into the entities defined in your GraphQL schema
Deploy the subgraph
Now all you have to do is to deploy it to the hosted service and let it sync with the node. After a while you can answer your query in less than a second using GraphQL. And if it needs refining, tweak your subgraph and redeploy it in one command line.
Summary
The Graph definitely met our expectations and fulfilled its promises. It’s easy to set up, customize and deploy. You don’t even need any kind of infrastructure if you decide to use their hosted service. On the other hand though, it may take a while to fully scan the node depending on how heavy your subgraph is.
If you wish, our CryptoKitty Subgraph is available in The Graph Explorer here:
Finally, we would like to give a big shout out to the team of The Graph, and a special thanks to Jannis Pohlmann and David Lutterkort for guiding us in optimizing our Subgraph and for their blazing fast reactivity when it encountered some sync issues (indexing Cryptokitties is actually pretty intense).
All Rockside updates
Subscribe to our newsletter.
Try Rockside for free, visit our website
Follow us on Twitter.
Join our Slack.
