7 reasons why volume and liquidity metrics are a concern in the CEXs industry
At Koinju, we want our indices and data aggregates to be compliant and representative within the meaning of IFRS accounting standards and BMR regulation. For this purpose, we’re currently establishing a transparent, robust and accurate framework to classify our crypto data sources according to their level of trustworthiness. Market manipulation and liquidity issues are part of these problematics we aim to verify among the different criteria of this selection framework.
Main issues with crypto metrics
Many crypto-industry participants do liquidity analysis without questioning their “methodology”. Indeed, some reports have been published presenting volume-based indicators giving entire confidence to all data sets provided by centralized exchanges. In fact, there are several problems with the methodology: more precisely, indicators are biased and too shady.
1-Trusty exchanges - First of all, it’s difficult to judge a volume or the depth of an orderbook by itself alone. The most common method is to compare to a reference pool of “most trusted” exchanges. These trusted exchanges are often chosen regarding legal, reputation criteria (e.g. Bitwise 10) and fame. Moreover, trust in these pools is commonly accepted because they have the largest market share. Here’s the first problem: it’s based on a biased decision to follow the common belief and it doesn’t fit our purpose.
2-Arbitrary - Secondly, the result of the analysis often depends on decision rules that are too arbitrary. If a metric reaches a certain threshold, analysts disqualify the volumes. For instance, we saw reports showing distribution of normalized volumes in which analysts consider that volumes above “mean +/- three standard deviations” were abnormal. Mathematically, this will always represent maximum 0,27% of anomalies and minimum 99,73% of consistent volumes because of the use of the normal law. Generalize the data using normalization (or any popular model) is always a two-sided decision. Some volumes are overestimated, and others are underestimated which have no sense (what’s the use case cheating with low volumes?).
3-Anomaly events and certification - Market manipulation detection methods are more often useful for detecting anomalies and outliers than for determining a common dishonest practice. We’re currently working on defining the best time step to use in our methodology and the best size range of the period to study. Most of the time, an anomaly is detected over a one-day period within one/five/ten minutes time steps. We don’t think it’s accurate to say that a market or a whole exchange should be disqualified permanently due to anomaly event.
4-Rate of fake trades - Finally, producing a fake trades rate is tricky. Indeed, it’s very complicated to be accurate and confident about this information. In practice, people who determine these “magic fake trades detection rates” use fairly-rough models and compare real data with the model. Generally speaking, the reported rates declared are very extremes and above all are expressed by exchange and not market by market. That’s why we better prefer indicators with a minimum of likelihood than giving that kind of absurd metric. Moreover, it doesn’t seem responsible as a financial market information provider to act like regarding for the efficiency of the market.
Issues with scoring
Data providers try to estimate the real volume thanks to liquidity or market manipulation criteria. We can see that kind of metrics on FTX Volume Monitor, CoinMarketCap Liquidity Score or CoinGecko Trust Score for example. Let’s for example dig into the FTX Volume Monitor :
5-Points and score attribution - If we examine the FTX methodology without questionning the consistency of the indicators — which aren’t all objective by the way — we realize that each criterion, if met, give a number of points weighted by criterion. Alameda have a weighting that is — here again — totally arbitrary. Then, they provide a score out of 5. Depending on the note, exchanges are classified into different groups, and they apply a coefficient to the volume reported by the exchanges. How to explain the choice of these coefficients?? Well, here’s the point.
6-Commercial impacts - The transparency approach is quite interesting given that FTX includes itself in this scoring. However, it is regrettable that it gives a classification of exchanges with a limited scope. Indeed, some users can be influenced by the ranking due to these arbitrary coefficients. Also, not providing a market-by-market analysis penalizes small and intermediate sized players who may have many markets that are much less active than the top 5. Applying an arbitrary weighting tenfold the power of exchanges with the biggest overall reputation. This favors the strongest in marketing without further detail.
7-Conflicts of interest - It is indisputable that the world of data providing is not free from conflicts of interest issues. The most popular platforms have business models mainly based on advertising (e.g. CoinGecko, CoinMarketCap). Among those who determine and produce these scores, these specific advertising-dependent actors play roles that are at odds with the role of independent score provider. There is a real question of legitimacy to be asked... FTX and Alameda Research are an exchange, market makers and also investors. Similarly, CoinMarketCap is owned by Binance. This market lacks independent and objective analysis providers.
To conclude, judging exchanges as a whole entity does not really make sense. It is important to separate and rate each market within these exchanges differently according to their own characteristics. To avoid the shortcomings above presented, indicators must be made with independence, qualified and accuracy. We look forward to presenting our progress to you and hope you will enjoy the way we do things.
