The Journey of a Quantitative Trading Strategy Development (Part 3)
A no-code overview
This article is the part 3 of a series of articles on quant trading. My goal is to present the steps involved in a quant trading project, in a simplified and non-programmatic way. On part 1 we defined our hypothesis, a market benchmark to which we can compare our trading results, and structured our data to use it in the following stages. Part 2 presented the initial backtests and some hidden costs in quant trading. The scripts and frameworks developed to this series are on my Github.
- Part 1 — Hypothesis and Benchmark
- Part 2 — Backtesting and Hidden Costs
- Part 3 — Key Portfolio Metrics
- Part 4 — Parametric Analysis and Overfitting
- Part 5 — Forward Testing and Statistical Significance
Disclaimer: This article was developed for informational purposes only, it is not a trading recommendation.
Disclaimer 2: All charts presented in this article were developed by myself.
After the previous steps we have a backtest framework and a strategy that performs better than the benchmark. But I’ll invite you to take a closer look at the chart of returns.
From June 2021 until June 2022 we can see a very critical moment in our strategy, where the portfolio value dropped from 8 times the initial investment to less than 4.
In other words, in a period of 12 months our algorithm lost more than half of what it had achieved in the years before. If the portfolio started with USD 100k we would be spending one entire year watching our balance go from USD 800k to USD 350k. Regardless of how much money we made with the strategy before that moment, it would be a psychological burden to keep it running during that period. You would probably make changes in the algorithm at some point, which could actually compromise the profits presented in the backtest at a later stage, or even shut it down. So how to measure our ability to operate a strategy and bear with losses when running a backtest?
Portfolio Metrics
Have you ever heard the saying that “the journey is more important than the destination”? Well, in quantitative trading this is definitely true. We might find an excellent strategy that provides outstanding returns, but if we have to go through hell to achieve them we will probably fail. In trading, consistence is key, and it is perfectly acceptable to trade off some returns to improve consistency. There are many metrics that can help us measure consistency and risk of a portfolio, let’s go through a few of them.
- Volatility
- Sharpe Ratio & Rolling Sharpe Ratio
- Maximum Drawdown
- Winning and Losing Days/Weeks/Months
- Mean Profit/Loss per Trade
- Number of trades
- Value at Risk (VaR)
Volatility
Volatility is the term given to an asset’s price variation. When the price is changing fast and frequently volatility is said to be high, and when those changes are not so significant it is considered low. People often think about volatility only when prices fall, however it can also refer to price rises too. Volatility is frequently used as a proxy for risk, but it is a necessary aspect for trading. It is so necessary that when price swings are small, traders have to leverage their operations to obtain returns from the market. Many books refer to volatility as the standard deviation of price changes in a period of time, but more important than calculating the standard deviation is to know the shape of the price change distribution. That will tell much more about market behavior than a simple standard deviation. Let’s check the hourly price change distribution for our study.
As we can see the price returns distribution is very high close to zero and has very long tails, which go much longer than the limits of this chart. We can also see in the chart a normal distribution with the exact same standard deviation of the actual price distribution. Can you see how different they are? The real price distribution has a much higher kurtosis, which means that we can expect many values close to the mean but also some very extreme values. That is why it is always important to explore the data instead of just following steps, understanding our data will allow us to understand the metrics that come from it.
Sharpe Ratio
In part 1 we discussed how traders look for an alpha in their operations, an excess return to the market. Sharpe Ratio is a metric that presents the relation between the excess return that a portfolio is obtaining and the risk to which it is exposed. The idea behind it is to compare the excess return per unit of risk, so that traders and investors can compare different assets (or different trading strategies). In many cases it is not worth to expose the portfolio to a much higher risk to obtain marginal excess returns. Let’s see how to calculate the Sharpe Ratio:
If you are an investor deciding where to allocate your money, there are different types of (quasi) risk-free assets that you can choose when calculating sharpe ratio. Even tough a long-only position on BTC is far from being risk-free, that is the market benchmark that we defined for our study, so when calculating excess returns we will subtract the returns of a long-only strategy from our Moving Average strategy.
When analyzing portfolios’ behavior in time it is very common to monitor the rolling sharpe ratio, which will apply that formula over a rolling period of time, for example, the past 60 or 90 days. In the chart below we can see the Cumulative and 90-day Rolling Sharpe Ratio calculated for our strategy. We can see that there is no consistency in the portfolio since the sharpe ratio is swinging a lot for a reasonable time window, but the cumulative metric always stays above 0.
The decision on how much is a reasonable excess return for unit risk is very individual, but this is a good indicator to compare different strategies or parameters.
Maximum Drawdown
Maximum Drawdown is another important metric. It calculates what is the maximum loss obtained by the portfolio after a new all-time high. Let’s check this in a chart to better understand it.
The green line identifies the maximum return obtained for every moment of our backtest. The drawdown is calculated subtracting the current return from the maximum, and the purple line is showing the maximum drawdown obtained for every moment during the backtest. The maximum drawdown from our backtest was in the end of June 2022.
Small values of maximum drawdown are a very good sign of consistency in a strategy. It shows that even with the changes in market dynamics (that happen frequently) the algorithm was able to profit and did not face long or sharp moments of losses. As you can see, this is not the case in our backtest.
Winning/Losing Percentage
No fancy name for this one, but it’s one of my favorites. When analyzing strategies we have to take into account the effect of random events. Some backtests may show good results for a model that just had luck in a few sharp price movement events in the past and nothing else. But everyday is a new day in the market, and relying on few events to profit is betting on chance. A strategy that has a considerable higher number of winning periods than losing ones has a higher probability to have a real edge in it, and thus a strong potential for long term profits. Let’s see the winning percentage of our backtest based in days.
It is not looking great but it is not also terrible. In this case we have to remember that we are using a momentum strategy, which is not expected to change position in a daily basis. So let’s change the timestep from days to months and analyze the results.
Now we can see a better behavior. Most of the time during the backtest the strategy had more than 50% of winning percentage in a monthly basis. But it is not a great result as well, there would be some bad months when trading.
Mean Profit/Loss per Trade
The mean profit and loss per trade also focuses in consistency. It can be calculated in a cumulative way, which will analyze the overall performance, or in a rolling way, which will look into recent performance. A winning strategy will have a higher mean profit than the mean loss, and a significant shift of behavior between the two can be the sign that the market dynamics is changing, what can be good or bad to your algorithm.
The same analysis can be done with the median instead of the mean. Applying a median will remove the effect of outliers and analyze how the strategy is performing. If the median loss is bigger than the median profit, it means that the algorithm is relying on especific events to obtain returns. That is perfectly fine in some strategies, however if it is not what you are targetting then some changes should be done.
Number of Trades per Period
There is not a right or wrong number of trades in an algorithm, but this metric is able to help your design. For example, the higher the number of trades, the more significant are the hidden costs that we discussed on part 2. So the excess return has to compensate this impact. Moreover, high-frequency strategies might require advanced infraestructure and coding to be actually profitable, otherwise we will not be able to reproduce the results obtained in a backtest. In some cases a high or low number of trades can inform us that our strategy is operating differently than what we expected, or that the parameters are not properly defined. For that reason we can monitor both cumulative and rolling count of trades, like in the image below, where the rolling counter is based on 90 days.
1200 trades in 56 months is around 21 trades per month in average. It seems a bit excessive for what we were expecting, so there is room for improvement.
Value at Risk (VaR)
VaR is an estimate of how much money can be lost before you can exit a position if price swings against you. VaR is calculated multiplying the size of a position by an estimated critical price swing, which can be calculated in many different ways. One of the most common methods takes a percentile of the asset’s historical price change distribution to estimate a future variation. Take a look at the price change distribution chart once again, but now in a percentage basis. The red line that you see identifies the 5th percentile of the distribution, which means that all the events to the left of the line represent 5% of the total events. That value is -1.17%, or -0.0117.
Let’s assume that we can use this distribution for probability analysis, and use that value as a reference for our critical price change. Then our conclusion is that in a hourly basis we should expect price drops bigger than 1.17% in less than 5% of the cases, or less than 2 times a month. If you want to be more on the safe side you can pick another percentile, for example 1% instead of 5%. If you are holding a position of USD100k, your VaR5% would be calculated as:
VaR5% = -0.0117 x 100,000 = USD1170.00
As I mentioned, VaR is also a parameter since traders limit their position size based on a VaR. So when backtesting different ideas it is important to limit the position on a especific VaR to make sure that when trading you will be able to hold a similar position as well. Also, the price distribution chart can be very different depending on the timestep that you’re analyzing. We have to make sure we choose a timestep in which our trading algorithm can exit positions or we will be miscalculating VaR.
Market volatility has its moments of highs and lows, so many traders prefer to assess VaR in a rolling form, analyzing only recent price swings and the current portfolio balance. The image below presents both a 90-day rolling VaR as well as the cumulative value for our backtest. As we can see the rolling value oscillates much more, which might require you to adjust portfolio positions more frequently.
Both methods can lead us to over or underestimation of market risks, if not carefully analized. After all, it is our responsibility to tune our models to reflect the market conditions, before using them to trade. Another important observation is that the VaR estimates the risk associated with sharp price movements in one timestep of our backtest, to account for unexpected situations. It does not estimates how much our model can lose by taking consecutive bad decisions, that is a task for the other metrics that we discussed.
Moreover, when the portfolio consists of more than one asset, each asset has to have its VaR calculated, and the final portfolio VaR will depend on how correlated those assets are. If they are perfectly correlated, the final VaR will be the sum of the individual components. If they are negatively correlated, one position is actually hedging the other and the final VaR is reduced. When there is a high diversification in low-correlated assets, the portfolio risk is significantly reduced, and calculating portfolio VaR in that case usually involves a stochastic method such as a Monte Carlo simulation.
There are hundreds of metrics that can be helpful when backtesting and trading, we really just scratched the surface. But now we have some basic tools to help us better evaluate our strategy, and from this analysis I would say that there is room for improvement. On part 4 we will work on that. Feel free to contact me on LinkedIn for any questions or suggetions.
