Do Not Follow The System
The meme out there is to “follow the system”. But you should follow a system as long as the underline market conditions that have contributed to its profitability persist. If not, you abandon it as quick as you can.
This is of course another useless generalization by those who do not understand systems but just repeat what some self-proclaimed experts have said usually in an effort to intimidate their audience.
There are two big lies regarding systems and this includes both trading and investing:
- Over-fitted strategies are unprofitable
- Discipline in following a system is the key to success
The first lie is told by academics with little or no skin-in-the-game. Academics want to sell mathematical analysis and attacking optimized systems facilitates this. There are many papers about over-fitting by people who lack understanding of trading and systems. Not only over-fitting may not be bad but not using over-fitted strategies may result in lower performance for an extended period of time.
I will attempt to decipher the convoluted logic here: academics usually look whether a system over-fits noise. If it does then according to them it is not a good system. But there are several problems with this approach. I will skip mathematics here and just mention the fact that some systems do not attempt to fit to noise and are not affected by it. There are systems that in general exploit inefficiencies in price action and as long as these inefficient exits, they remain profitable. But these systems appear over-fitted post hoc.
Do we care whether these inefficiencies are called noise by academics? No, we do not. If we care, we may end up sitting on the sidelines for many years while the academics get tenure publishing papers about these “random” effects.
I have included specific examples in my paper “Limitations of Quantitative Claims About Trading Strategy Evaluation” about optimized systems that performed well for an extended period of time because the underline “noise” or “inefficiency” persisted during that time. Actually, some CTAs in the 1980s made huge money from trading optimized and even over-fitted trend-following systems due to persisting market conditions. Had those CTAs listened to academics they would have not traded those systems because the tests would have rendered them not statistically significant.
Below is an example of a simple moving average system in S&P 500 in daily timeframe since 1960. A long position is established if price is above the 25-day moving average and it is closed if price falls below the average.
This simple system was data-mined from 1960 to 1975, tested out-of-sample from 1976 to 1984 and found profitable and then was traded for 13 years with excellent performance, as shown in the above chart. Then, the system suddenly stopped working after 1999.
The optimized, or even over-fitted, system above stopped working in 1999 because market conditions changed and the inefficiency that contributed to its profitability vanished. For more details click here and here.
Many academics, instead of dealing with the difficult problem of changes in market conditions, or regime switches, concentrate on the easy but even wrong problem of over-fitting. They have succeeded in shifting the attention of many traders and even professionals to that often at a huge opportunity cost but they have secured positions and tenure in the meantime.
Academics have little to say about the problem of changing market conditions. Usually that is the subject of experienced traders with skin-in-the-game. All statistical tools in the world will not help if there is lack of experience.
To summarize, academics have shifted attention to the wrong problem in trading system development because the more important issue is not over-fitting but changes in market conditions for a large class of trading systems. Obviously there are systems that are fitted to noise over short periods of time and then fail immediately but that does not apply to most systems tested over many decades, as in the example above, where the system was tested for 25 years in combined in-sample and out-of-sample.
We now move to the second part regarding the “follow the system” meme.
As it may also be seen from the above example, even if a system has worked for almost 40 years it can suddenly stop working. Again, this is not because the system was over-fitted or optimized; it is because a drastic change in market conditions has occurred. Therefore, during certain times, “do not follow the system” is the right meme.
Discipline is required but in following the right system. Many died trying to conquer Mount Everest because they were disciplined and went against their own limitations. Being disciplined can be tragic when there is a paradigm change. I believe this is where the markets may be at this point. Many trading strategies will stop working and generate losses for many years into the future. “Follow the system” under changing market conditions is equivalent to those naive cryptocurrency traders recommending “hodling” when bitcoin was at $18K in a hope of avoiding the collapse to current price of about $8K.
One strategy that may withstand a regime change is market neutral long/short equity. I have concentrated in developing tools that allow development of these strategies in the last three years based on other tools I have developed before that. Even if a major regime shift does not occur, it may pay to have a certain allocation to those strategies. These strategies tend to underperform during strong bull markets but they are usually robust through sideways and down markets.
Finally, the academic and trading/investment mix is not a very healthy one. For many years academics have tried to convince everyone that markets are random while it was obvious they are far from that. Now some of them focus on over-fitting and invent convoluted ways of evaluating strategies when they should be concentrating instead on forecasting market condition changes. As I have demonstrated with a simple example, these wrong approaches can be costly. However, academics have managed to take out of the loop the experienced trader and strategy developer and replace it with convoluted statistical approaches that in many cases render 99% of trading strategies insignificant. But the fact is that many have made a lot of money in the past with seemingly statistically insignificant strategies so obviously academics are knocking on the wrong door here.
If you have any questions or comments, happy to connect on Twitter:@mikeharrisNY
This article was originally published in Price Action Lab Blog
About the author: Michael Harris is a trader and best selling author. He is also the developer of the first commercial software for identifying parameter-less patterns in price action 17 years ago. In the last seven years he has worked on the development of DLPAL, a software program that can be used to identify short-term anomalies in market data for use with fixed and machine learning models. Click here for more.