Tracking realized historical yield from DeFi protocols
Have you ever deposited in a DeFi protocol that promised 923% APY and had underwhelming returns? That’s why we use every trick in the book to create DeFiReturns.com, a database of realized yield in DeFi.
Why?
Degens won’t think twice before aping into a protocol that promises infinite APY. However, without any type of oversight on what is valid in DeFi marketing, builders without a moral compass may be tempted to show numbers that are, let’s say, excessively optimistic.
The transparency of data is one of the critical features of Ethereum, so theoretically, anyone can access the earnings history of a given protocol to verify if the APY displayed on the website is true, but this requires understanding the smart contract, accessing the RPC provider, reading the desired fields, and cataloging them.
Our desire to democratize the ability to extract, visualize, and compare each protocol’s returns led us to publicly release an internal tool that was used to create a chart displaying the historical earnings of our vault.
What?
Although other applications display the yield from various apps, we could not find any that offered the ability to 1) compare strategies’ performance side-by-side, 2) choose the timeframe for comparison, and 3) collect solely on-chain data. Therefore, we aimed to build a website where you could clearly see how much you would make, net of fees, if you deposited in a protocol, and where you have all the options above.
Plotting the accumulated yield from a strategy helps you understand if the marketed APY on a protocol’s website is real and consistent. It can also give you insights on whether or not your token count (principal) is protected or not. Putting these charts side-by-side will reveal how similar strategies compare.
Choosing the timeframe for comparison allows users to zoom in and out of selected time periods, analyze the behavior in different granularities in detail, and fit the plotted data to the moments they were exposed to the strategy.
How?
First of all, you’ll need access to an Archival Node to read historical data from the chain. That’s easy because RPC providers like Infura or Alchemy offer generous free tiers that you can use.
The next step is to understand how to use the RPC Provider to access historical data. Archival nodes expose an HTTP API that you can use to interact with the blockchain, and one of the most used methods is eth_call. With this method, you can call a view function from a contract, select the desired block number, and even override state variables if needed. But that's an advanced use case. Let's stick to reading real values for now. Alchemy provides pretty decent documentation on how to use this method at https://docs.alchemy.com/reference/eth-call.
If you feel hardcore, you can encode the function call using the contract ABI, generate the calldata, and pass in the data field of the eth_call method. If you are like me and, most part of the time, you are feeling like a little princess 👸and all you want is a nice dev experience, you can use ethers.js that does all the hard work, leaving you with something as nice as:
const lendingPool = new ethers.Contract(lendingPoolAddress, abi, rpcProvider)
lendingPool.functions.getReserveNormalizedIncome(usdcAddress, { blockTag: 16000000 })In our first prototype, we wanted a generic approach to assessing the protocols’ performance without having to write new code for every tracked protocol. We started by finding investors that deposited early in the project and simulating a withdrawal action every N blocks.
It worked well for a while, but we knew that people would change their positions sooner or later, and our tracker would show crazy numbers or simply crash. While researching how to make something more robust, we found out that most protocols have some sort of index that tracks their yield. We then chose to take down the “no new code” requirement and replace it with a “one snippet for each protocol” approach.
Pools that work with rebasing tokens, like Lido, or those that utilize conversion rate tokens like AAVE, are super easy to track. Other pools can be more challenging. Let’s explore how to track the USDC pool on Aave, the ETH pool on Compound V3, and Opyn’s Zen Bull strategy.
USDC on Aave
Let's start by analyzing the LendingPool SC from Aave.
As you can see, AAVE exports this view function, making it easy for anyone to check how much interest has accrued. Tracking returns for lenders on Aave is as simple as reading this variable and comparing it with previous values for each asset.
const example = async () => {
const lendingPoolAddress = '0x7d2768dE32b0b80b7a3454c06BdAc94A69DDc7A9'
const usdcAddress = '0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48'
const rpcProvider = getProvider()
const lendingPool = new ethers.Contract(lendingPoolAddress, abi, rpcProvider)
const reserveNormalizedIncomeNow = await lendingPool.getReserveNormalizedIncome(usdcAddress)
const reserveNormalizedIncomeBefore = await lendingPool.getReserveNormalizedIncome(usdcAddress, { blockTag: 16000000 })
console.log(reserveNormalizedIncomeNow.toString())
console.log(reserveNormalizedIncomeBefore.toString())
const index = new BigNumber(reserveNormalizedIncomeNow.toString()).dividedBy(new BigNumber(reserveNormalizedIncomeBefore.toString()))
console.log(index.toString())
}With this snippet, you interact with the Lending Pool smart contract to read the return value of the function getReserveNormalizedIncome at block number 16000000 and in the latest block. Dividing these results gives you the index value, which shows how much more tokens one would have if they deposited USDC on Aave at block 16000000.
Compound V3
Fetching data on Compound v1 and v2 is as easy as doing this on AAVE. On the third version of this lending protocol, however, the developers decided to make some variables private. For this reason, there are no easy roads to the data you want. On the other hand, private variables don’t mean secret variables, so let’s take the off-the-beaten-path approach to get our metrics.
If you can’t use eth_call to call a view function on-chain or to read a public variable, fear not because you can always read storage variables slot by slot. You can select the slot of interest, ask for its contents, decode the bytes you’ve received, and you’ll have the data you need. Let’s see it step by step.
Reading the SC code, we knew that we’d need some public variables (that’s fine, we already know how to do this), but some other variables are private, like these:
The solidity compiler packs variables in slots of size 32 bytes following the declaration order. To accurately calculate Compound’s v3 yield, we need to know the lastAccrualTime and the baseSupplyIndex. We can use ethers.js to access it with these functions:
const encodedBaseSupplyIndex = await rpcProvider.getStorageAt(address, 0, blockTag)
const encodedLastAccrualTime = await rpcProvider.getStorageAt(address, 1, blockTag)It’s not always the case that finding the right slot number is as straightforward as shown in this example. Things can get pretty complex pretty fast, so if you have issues finding the right number, you can find the docs at https://docs.soliditylang.org/en/latest/internals/layout_in_storage.html.
Now that you have the slots, you still need to decode their content. For this task, there’s an alpha leak: bytes are encoded in reverse order and offset by the variable sizes that precede them. With this golden piece of knowledge in hand, ethers.js comes to the rescue again and exports the utils:
In our case, this is how we use these utils to read our desired variables:
// Decoded values are segmented from right to left
// baseSupplyIndex:uint64 has 8 bytes and offset from right to left 0 bytes
// since it's the first variable of slot 0
const baseSupplyIndex = getStorageVariable(encodedBaseSupplyIndex, 8, 0)
// lastAccrualTime:uint40 has 5 bytes and offset from right to left 26 bytes
// since it comes after 2 uint104 variables. uint104 + uint104 = 208 bits = 26 bytes
const lastAccrualTime = getStorageVariable(encodedLastAccrualTime, 5, 26)
function getStorageVariable (storageData: string, dataSizeInBytes: number, offsetInBytes: number): BigNumber {
const STORAGE_SIZE = 32
const actualStorageSize = ethers.utils.hexDataLength(storageData)
if (actualStorageSize !== STORAGE_SIZE) {
throw new Error(`\`data\` is not ${STORAGE_SIZE} bytes, received: ${actualStorageSize} bytes`)
}
const sliced = ethers.utils.hexDataSlice(storageData, (STORAGE_SIZE - offsetInBytes) - dataSizeInBytes, STORAGE_SIZE - offsetInBytes)
return new BigNumber(sliced)
}Now you not only are a little bit smarter, but you also have all the data needed to implement your bread-and-butter business logic to find Compound’s returns :)
Zen Bull
After a long period of inspection, we couldn’t find an easy way to fetch public variables and code some function to calculate Zen Bull’s returns. The easiest way would be to read data from an event emitted during the withdrawal.
Thank God tracing-enabled Geth nodes expose a custom RPC method: debug_traceCall lets you simulate a function call and returns not only the return values but also the events emitted! If you want to learn how to do it, check this answer on SO.
You can also pay Tenderly and use their awesome Simulate API :)
That's it for today, folks! I hope you enjoyed the text. Please visit DefiReturns.com to check the final result, and feel free to DM me any questions on Twitter @LeoPapais. See ya!
