Retrieve and Analyze DeFi Data With Keyko’s Python Library

Simplify DeFi data aggregation thanks to our python API

Introduction

In our previous post, we talked about the existing problems around data retrieval in DeFi due to the lack of standards, as well as the many transformations needed to create a dataset, repeating data transformation and complex acquisition processes. Thanks to The Graph’s grant, we were able to set about tackling this challenge.

In this post, we are going to explain what we built to solve this problem to allow easy data aggregation using just a python API and how this library can be used to retrieve and analyze data.

How the library works

Our main objective in creating this library was to create software flexible enough to be able to adapt to the dynamic DeFi world and make it easier for the community to add functionalities and protocols in a simple way.

To do so, we have created a library as configurable as possible and based on configuration files to add new entities or new protocols to the library with minimal changes in code.

The library retrieves and transforms the data in three different steps:

• Retrieving the data: The user calls the library API to get data from a specific protocol in a specific event type. The library finds the fields that are needed to query the subgraph in a configuration file and sends the query to the endpoint.
• Map the data: With the response from the service, this data should be mapped in a common structure that all the protocols have, so in the next step the library converts the fields from the subgraph in a common structure to return to the user.
• Transform the data: In some cases, one or more fields from the subgraph should be transformed, for example, convert the units from wei units to decimal format, or replace the event id to get only the transaction id. This final step is done once the data is mapped, and the transformations functions are also called dynamically depending on the configuration written in the config file.

Once that these three steps are done, a common and structured dataset is returned to the user.

How to use the library

Now that we know how the library works, we can see an example of how to use it to get data from several protocols.

First, we need to know which protocols are currently supported, this is in our supported protocols section of the repository. More protocols will be supported in the future.

Now we can create an object for each type of protocol (Lending, Dex or Oracle). Let’s see an example.

To create the objects, we need to pass as an argument the name of the protocol, the blockchain in which protocol is deployed, and the version.

And once these objects are created, we can query from all of them to get all the events of a specific type. In this example, we are getting all the borrow events in the last week.

If we pass this result to a pandas data frame, our dataset is already created.

And now that the dataset is created, we can start to ask questions about the data, and use our favorite python plot library to plot the results.

What is the protocol with the most borrow events adding all the blockchains?

In which blockchain are there more events in this period?

Which is the most borrowed token in these events?

As you can see, now that the dataset is created, all the typical data analysis can be applied, and not only for lending, also for Dex and oracles. Check out the docs and examples for more detail.

Contributions

This library is open-source, so if you think that your favorite protocol should be supported or another type of event should be added, take a look at the contributions section in the docs to submit a pull request and add new functionality to the library.

We hope that this library allows more people to get and analyze DeFi data, and create and more open and transparent financial world.

About Keyko

Keyko is a Web 3.0 solutions provider offering integrated decentralized solutions and advisory for enterprise and startups. We thrive on providing organizations and individuals insights into this emerging digital world where capitalism and ownership are being profoundly rethought.

If you would like to know more about our services or are interested in having a chat please reach out to us on info@keyko.io or visit our website here.

If you enjoyed this article let us know by clapping or leaving your comment below.