Are the quants doing the right thing?

“Past performance is not an indicator of future results” we all read this sentence on footnotes of financial products releases. In a world where automated strategies are playing a large role in trading, a concern may soon rise about the quality of quantitative trading strategies.

Recently, I came across a message that tries to explain why the quality of trading strategies may be misleading and it may be a systemic issue around the industry.

“When investment advisers do not control for backtest overfitting, good backtest performance is an indicator of negative future results.” [1].

What this says is that optimization over historical data without proper out-of-sample testing is a recipe for negative returns. Overfitting can be defined as the process of fitting a model to the data points. It occurs due to over optimization and parameter tuning, and when it happens every model looks great. A good model should learn the structure of the data not its data points.

Young quantitative traders should take the role of testing the results of their optimization on out-of-sample data seriously. Even doing so, they should judge whether they are doing it right. They should be even more concerned if they are dealing with strategies based on technical indicators, especially if they show good max drawdown and sharpe ratio. It is pretty easy to find a fantastic model when you have too many parameters to tune. It is like looking to the sky trying to find a cloud that looks like a rabbit, you will find it.

Trading strategies based on technical indicators tend to have several parameters that are optimized over long simulation on historical data. It is pretty unlikely that the combination of technical indicators will generalize a trend upwards / downwards. Although they can be profitable for while, no one can predict the development of a price curve. You can have 100 good trades in sequence and a single trade may eat all your previous profits.

At Algologic.ai, we are still learning how to deal with technical indicator based strategies. What we learned so far is that they work, but not always. Knowing to classify when they work makes the difference. To avoid dealing with issues of overfitting, we based our most successful strategies on pattern matching and information theory principles. They give us a better tooling for spotting structures in the data that will repeat in the future. Of course there is the stochastic nature of the pricing data, and success is never guaranteed, uncertainty cannot be eliminated.

“Prediction is hard, especially about the future.”[several authors].

Quants should be suspicious about trading strategies that show good past performance based on technical indicators. They may be over-fitted to the data. The fascination for algorithms and AI may be blurring the reality, algorithms will do what they are designed for doing. If they are badly designed / trained, they will do the wrong things.