Maker Capital at Risk: Model Methodology
Simulating collateral price shocks’ effect on protocol economics
Introduction
In our previous post, we explored the reasoning behind building the Maker Risk Dashboard and how it’s used for Computer-Aided Governance. In this post, we dive deeper into Collateral Risk Model which is used to propose Risk Premium and Debt Ceiling, two key governance risk parameters. This can then be used to derive Capital at Risk, a portfolio level metric. It estimates potential protocol losses based on historical data, present protocol state and conservative assumptions about potential market price shocks.
Given that this post focuses on building a risk model of a specific part of the Maker system, it is assumed that the reader already has a basic understanding of its mechanisms. Nonetheless, before diving deeper into the model, we’ll give a short refresher around MakerDAO and its governance.
MakerDAO provides the service of overcollateralized loans through financial instruments called vaults. Users lock their assets as collateral to mint debt in units of DAI (which is pegged to USD). If there’s a major drop in collateral price which can cause a vault to be liquidated, MakerDAO can experience a protocol loss because of bad debt. This happens when the value of the vault’s locked collateral asset(s) that are auctioned off during a liquidation is lower than the vault’s outstanding DAI. In the worst case scenario if there was too much bad debt accrued in the protocol, Maker Protocol would become insolvent.
For this reason it is crucial to understand how the current Maker portfolio (collection of all vaults across different vault types) would be affected by various levels of collateral price drops. That’s where Collateral Risk Model comes in.
Collateral Risk Model
The Collateral Risk Model was first introduced in April 2020. Since then, it has been continuously improved and adapted to both ecosystem and protocol (e.g. Liquidations 2.0.) developments.
The model iterates over different amounts of simulated Debt Exposure and computes the output given the chosen inputs. We’ll explore the model outputs first and give context around why they are necessary for protocol governance. Then we’ll dive deeper into the underlying methodology with the description of model inputs.
It’s also worth mentioning that all our proposals are shared publicly are not binding in any way. Just like with any other DAO contributor, they need to go through the formal governance process to be enacted by the MKR voters (MakerDAO community).
Risk Premium
Given the model’s estimated protocol losses during these simulated market shocks, we then use this information to compute a Risk Premium. The Risk Premium is used together with other factors for our team to propose a Stability Fee for vault services (DAI debt). The other factors include competitive landscape and the general elasticity of vault owners to the changes in Stability Fees. For the protocol to be economically sustainable, it needs to charge more for its services via the Stability Fee than estimated protocol losses in a projected time period (usually 1 year). Risk premium at the vault type’s current debt exposure is then used to guide us on the proposed Stability Fee.
Debt Ceiling
The model also outputs Debt Ceiling, another parameter value. It’s the maximum amount of debt that MakerDAO community is willing to generate from a specific collateral type (outstanding DAI exposure). This is again context-dependent with some factors being more quantitative (eg. liquidity risk) and some more qualitative (eg. potential governance risk via multi-sig wallets).
If slippage is high with relatively small trade amounts in chosen collateral assets, we need to be careful with raising the debt ceiling parameter too much. This matters in case of large debt size liquidation where locked assets are auctioned off, causing a substantial slippage and potential bad debt. Similar caution needs to be applied with more qualitative factors.
There are other parameters specified per each collateral type but as these rarely change and are not an output of the Collateral Risk Model, we will not dive deeper into those.
The model determines the Debt Ceiling when the Risk Premium reaches 10%.
Methodology
Now we understand why we need the Collateral Risk Model and that it should output two crucial parameters for Maker governance: Risk Premium and Debt Ceiling.
The next question is: what are the inputs that should have an impact on the model’s output?
The first input is the jump severity: severity of the price drop that happens in a specific collateral asset. The more severe the drop, the more debt is likely to be liquidated with more potential bad debt to accumulate. By default, we currently use 45% for WBTC, 50% for ETH, and 60% for all other volatile assets. This is used to estimate a potential market shock. It can also be seen as either a previously unseen intra-day drop (tail risk) or a more commonly seen multi-day drop where a certain proportion of the vault owners wouldn’t be protecting their positions. But considering that this input can be considered subjective, anyone can use the interactive simulation model and observe the changes in the output. We chose the default values based on observed historical price drops, as an example you can see the WETH OSM price drawdown chart below. We intentionally chose a simple heuristic when modeling this parameter to maximize the potential community involvement and its understanding of key model levers. Intuitively, the higher the chosen jump severity, the higher the computed Risk Premium.
The second input is the jump frequency: frequency of the price drops in a specific time frame. Similar to a VaR calculation, we want to understand how much a specific number of market shocks within a year would affect the protocol economics. By default, this parameter is set to 2, estimating 2 major price shocks within a year. Similar to other parameters, this is intentionally conservative. Intuitively, the higher the chosen jump frequency, the higher the computed Risk Premium.
The third input is keeper profit which determines how much financial incentives participants in collateral auctions receive. This is by default set to 5%. Intuitively, the higher the chosen keeper profit, the higher the computed Risk Premium.
The fourth input is the share of vaults protected. This is the estimation of the percentage of debt that will not be liquidated during price drops due to vault owners’ active management of their positions. This one is different from the other three given that it is not a manual input by the model’s user. It is a separate model that calculates the parameter value split per vault type and collateral price drop by looking at historical patterns of its individual vaults. Intuitively, the higher the chosen share of vaults protected, the lower the computed Risk Premium.
Aligned with our modeling principles, we aimed to create a methodology behind the share of vaults protected parameter to be explainable when we noticed some changes in the model output. Intuitively, the higher the Collateralization Ratio (outstanding DAI relative to collateral asset value) a specific vault has, the higher the price drop it could experience with no protection of its position and still not be liquidated. If this is the case with a vault given the jump severity percentage, then it is automatically scored as protected. What about vaults with a lower Collateralization Ratio, especially if they were well protected when looking at the historical data? A vault that managed to protect its position during past price drops should be deemed safer and less likely to contribute to bad debt accumulation relative to some other that was opened recently.
This is why we expanded the methodology to incorporate the Vault Protection Score, a score assigned to each individual vault. Each vault is assigned one of the three scores: Low risk, Medium risk or High risk. Share of vaults protected is then computed by accumulating DAI debt, split by individual Vault Protection Score. Even though this could be modeled as a machine learning problem, we decided to go with a heuristic as this can often provide a great model baseline to potentially improve upon later.
When determining an individual Vault Protection Score, there are multiple heuristics that we’re looking at. One example is determining an active vaults management quantified by the number of actions relative to the collateral asset price volatility in the last 30 days. We call this heuristic volatility actions. These are other heuristics, one of which is whether a vault is protected by DeFi Saver, a DeFi automation service that’s proven itself to be a robust solution for an active management of vault positions. While our methodology has been slightly updated since sharing it in Maker forum post, it can serve a good foundation to better understand how it works.
There are also a few high-level modeling mechanics that we implemented to better approximate how the real-world system works. One of the simpler and crucial ones is inter-connectedness across different vault types collateralized by the same collateral asset. If ETH price drops, all of the vault types collateralized by it (ETH-A, ETH-B and ETH-C) contribute to a total liquidated amount. This in turn increases the price slippage which increases the simulated protocol loss.
We also introduced a conservative modeling assumption that Share of Vaults Protected drops when our simulated Debt Exposure is higher than current Debt Exposure of a vault type. This is calculated proportionally when comparing simulated Debt Exposure and current Debt Exposure. This is why we can see a steep increase in the computed Risk Premium when simulated Debt Exposure is high.
In order to better understand model mechanics, we enable the model user to play with different parameter values. For tracking purposes and guiding our recommendations, we use default values set by our team.
You can see below a system diagram of the Collateral Risk Model. Much of complexity is abstracted away to focus on the high-level components.
Capital at Risk
There is another metric already mentioned above that is a risk view on a portfolio level and is derived from the computed Risk Premiums across different vault types: Capital at Risk. It is computed by multiplying each vault type’s Risk Premium with its outstanding DAI. This is then a high-level overview of the Maker system. If this metric changes, we can then look into individual vault type data in order to investigate what input(s) changed the outputs of the model. For monitoring purposes we use a 30-day rolling average to capture longer-term trends.
Currently, this is used to size the necessary Surplus Buffer because revenue from Risk Premiums is not instantaneous which presents a need to cover simulated protocol losses for a chosen period (in our case 1 year). Non-instantaneous nature of Maker revenue means that it is always in a projected, annualized form wheres the “looming” losses via different risk factors (eg. increased exposure) can occur before revenue actually ends up in the protocol’s balance sheet. Keep in mind that our team’s scope is limited to analyzing financial risk which means that other sources of risk (eg. technical, smart contract risk) is not taken into account which would also increase the necessary coverage of Surplus Buffer.
Modeling principles
We presented our three key modeling principles in our previous post: actionability, conservativism and explainability. Conservativism is a principle, embedded in the model setup and was already explained above. Actionability maps the model outputs to actual governance parameter decisions, in this case recommendation of Risk Premiums and Debt Ceilings. Explainability on the other hand needs to be shown in order to be understood on the model level.
Before we dive deeper into that, it’s worth explaining the reasoning behind our choice of model setup with the focus only on modeling system components that are relevant for our problem at hand. There are many valid points that can be made why moving towards an even more accurate representation (digital twin) of Maker would be necessary, running simulations on top of actual smart contract code. cadCAD and TokenSpice are both tools that are quickly maturing and could eventually support an alternative direction of using simulation modeling of tail risk scenarios with EVM-in-the-loop. In the case of Collateral Risk Model, we currently don’t see the increase in model complexity as essential as it is in some other simulation use-cases (eg. simulating complex attack vectors). Instead of porting what has proven to work in TradFi, we believe that it is crucial to create DeFi-specific modeling methodologies. This is why we created Capital at Risk metric which takes this into account. It’s also difficult to compare different models given that there is no clear objective function to justify additional complexity. Even if we somehow come up with one, complexity often brings additional subjectively chosen parameters which further damage the ‘improved’ model’s objectivity premise. We prioritized model explainability over better system identification which would most likely cause us to lean towards more black box modeling. Regardless, these don’t need to be mutually exclusive and we aim to tackle this challenge in the further model improvements.
We’ll continue through a demonstration of model explainability by identifying a difference in the model output and investigate the source of the change.
Explainability Deep Dive
The example below is a walk-through for investigating a recent decrease in computed Risk Premium on ETH-A vault type.
We begin by looking at Capital at Risk metric to see Maker portfolio-level measure of risky exposure. As mentioned above, it is computed as a 30-day rolling average to notice longer-lasting state changes in the portfolio. We also put it into perspective by comparing it to Total Risky Debt and Surplus Buffer. We can see that currently, Surplus Buffer is higher than computed Capital at Risk which looks healthy if we focus only on financial risk.
Below we go from a portfolio view to a vault type view. We can see how total Capital at Risk is split across different vault types and how it changed from the previous day. As expected, most of Capital at Risk comes from a couple of largest vault types. Similar to the previous element, this is meant to be scannable with the option to dive deep into specifics if we notice some major changes. We don’t see any significant (e.g. 2x) changes in the computed Risk Premiums.
Next, we look at historical Capital at Risk amount and see that there’s been a strong decrease in the last month or so. Given that ETH-A has the largest debt exposure, we decide to look further into its risk profile to see whether there’s some major change there. Whereas in the past, Surplus Buffer didn’t cover the entire Capital at Risk, this changed during this period.
There’s a few of metrics on the page, from basic ones such as the number of vaults, Total Debt Exposure, Stability Fee and Debt-Weighted Collateralization Ratio. There’s also metrics about vault exposure distribution where we see that 10 vaults contribute to 72% of total vault type’s outstanding DAI. There is also a chart that shows vaults debt at risk per specific collateral price drop. We can also see that most of vault type exposure is scored as Low risk.
What we’ll focus on is the Collateral Risk Model which contributes to changes in Capital at Risk. You can see in the chart below the model outputs which are maximum Debt Ceiling and Risk Premium at current Debt Exposure. Also, we can see vaults protected share parameter which is computed per each vault type and jump severity combination where we use the above vaults debt at risk data in the background. Vault type’s Capital at Risk is also shown.
There is also an option to click on ‘Risk model simulation’ where you can play with different model input values and observe how it changes the model outputs. This is a great way to get a better intuition behind the proportional contribution of different model input assumptions.
There are four most important signals that change the computed Risk Premium: total vault type debt exposure, collateral asset slippage, vault collateralization and protection score risk. That’s where we look first when we want to better understand the dynamics behind larger changes in computed outputs. We aim to monitor all these in individual, separate metrics. The first one is straight-forward. The second and third have their own derived metrics that we use to simplify the monitoring process. We’ll revisit the fourth later on for further explanation.
To look into collateral asset slippage, you can click on ‘Liquidity Score’ tab and get a chart that shows historical patterns. The metric computes the percentage of debt captured if we auctioned off the entire current Debt Exposure.We chose this value to monitor collateral asset’s slippage over time given a certain value swapped on a DEX, not to approximate how much actual value could be liquidated practice. We also have an asset slippage page for WETH and other collateral assets in which you can compare on-chain dynamics and also more general CEX activity metrics. We currently use 1Inch as a proxy for DEX slippage. If Liquidity Score increases,we can expect Risk Premiums to decrease and maximum Debt Ceiling to increase.
The metric to monitor vault collateralization is available in ‘Collateralization Ratio’ tab and is computed by debt-weighting individual vaults’ Collateralization Ratios. If this metric value increases, we can expect Risk Premiums to decrease and maximum Debt Ceiling to increase.
We can also have an overview of how the computed Risk Premium changed over time. That’s accessible via “Historical vaults stats” section. Aligned with a more general decrease in Capital at Risk, we can see that the computed ETH-A Risk Premium has also decreased in the past month.
When looking at underlying dynamics, we see that the first signal explaining this is an increase in Debt-Weighted Collateralization Ratio. This is aligned wit the decrease in the computed Risk Premium.
Liquidity Score also shows an increase in the last month which is also aligned with the change in model output.
While we haven’t seen any major difference in total vault type exposure in the last month, we can see that given the drop since November last year, this also contributed to vault type’s decrease in Risk Premium and consequently also on the decrease in Capital at Risk.
Activity of individual vaults, especially whales, give us an more in-depth view of changes in total debt. ETH-A’s largest vault (8463) owned by Nexo was a large contributor to both in the rise in Debt Exposure last year and also more recent decrease this year. We also created a whale monitoring page for this purpose.
The last signal mentioned above is protection score risk. It’s computed by aggregating daily debt exposure over assigned Vault Protection Score for each individual vault. This enables us to track the movement of vaults, especially larger ones, between different protection scores if their behavior and/or vault state meaningfully changes. It also shows general distribution at a given point in time, whether most vaults are well protected or not. This is especially important in the case of MakerDAO where vault owners have opportunity to protect their vaults because of the OSM price delay.
The conclusion from the above signals is that ETH-A has a couple of contributing factors to the decrease in the computed Risk Premium. Debt Exposure decreased, largely because of a whales repaying their DAI debt after the price shock in the end of January. The other two factors are both an increase in Debt-Weighted Collateralization Ratio and Liquidity Score. If we look at other vault types we see similar patterns which when aggregated, show a pattern why Capital at Risk decreased in this period. We were able to validate why Capital at Risk amount changed and concluded that the current Surplus Buffer is sized well given the protocol financial risk.
Beyond the above mentioned signals and their impact on the model output, there are also other underlying metrics which have an effect on the signals themselves. The chart below shows that the decrease in protection score risk mostly comes from vaults assigned a Low risk score which is in our case decreased exposure from whales shown above (Nexo’s vault on its own contributed to a decrease of 700M). This in turn impacts share of vaults protected per vault type which is an input in the Collateral Risk Model. If this would come from either High risk or Medium risk vaults, the decrease in ETH-A Risk Premium would be even higher.
There is also an option to look at individual vaults and their stats, from the collateral price at which a vault would be liquidated given the current Collateralization Ratio to the assigned Vault Protection Score. We also track DAI capital which shows how much available funds the vault owner has at hand in their wallet to repay their debt in the case of a price shock. This is an important metric for our team to monitor considering that it can be a strong indication of how well protected is the Maker portfolio. You can also see the protocol-specific tags which show in which other protocols the vault owner holds funds. Among other activities, this can be useful to track vault owners’ tendency to use recursive leverage strategies in other protocols (which decreases financial protocol risk given specific outstanding debt).
There are other sections in the dashboard that are beyond the scope of this post. We’re inviting the readers to dive into those themselves, hoping they’ll find it useful. We’re also regularly adding new functionality to continue serving MakerDAO’s community needs.
Conclusion
This post presented Capital at Risk model methodology and how we use it to support Maker governance processes with risk parameter recommendations.
Our aim is to continue building models that approximate real-world crypto-economic systems as much as possible without suffering the price of any of the above mentioned modeling principles. For that reason we are bootstrapping our risk-related activities to build a more robust DeFi ecosystem that can survive the test of time.
Acknowledgements
Thanks to Primoz Kordez, Peter Hacker and Kurt Barry for reading earlier drafts of this article and providing valuable feedback.
