Backtesting 101: How to accurately evaluate your trading approach? Part 1
Backtesting is an essential part of developing a successful day trading strategy. By simulating the performance of your strategy in past market conditions, you can identify its strengths and weaknesses, and make any necessary adjustments before risking your capital in the live market. This can help you improve your chances of success and increase your profitability as a day trader. However, in order to do that, you need to start with the right foundation.
In this three-part article series, we will explore everything necessary for you to know to backtest your strategy correctly. Before you start thinking about backtesting, you need to determine how you want to trade. I wrote about trading strategy here.
General testing plan
Here are the general steps to follow when backtesting your trading strategy:
- Collect historical data: The first step in backtesting is to collect historical data for the asset class you want to trade. This data should cover the time period you want to test, and should be as comprehensive and accurate as possible. As mentioned — cleo.finance is one of the platforms that does this step for you.
- Set up your backtesting platform: Once you have your historical data, you need to set up a backtesting platform. This can be a dedicated backtesting application — such as cleo.finance or TradingView, or a spreadsheet program like Excel. Your platform should be able to load historical data and apply your trading rules and indicators to simulate the performance of your strategy.
- Test your strategy: Once your backtesting platform is set up, you can start testing your strategy. This involves applying your rules and indicators to the historical data and seeing how your strategy would have performed in the past. You can then analyze the results of your backtesting to identify any strengths and weaknesses in your strategy. This process can be fully automated in cleo.finance, or to certain degree in other platforms as well.
- Make adjustments and continue testing: Based on the results of your backtesting, you may need to make adjustments to your strategy. We call these the Optimization Loops. This could involve changing your trading rules or indicators, or altering your approach to risk management. Once you have made these adjustments, you can continue backtesting your strategy to see how it performs in different market conditions.
Now that we are armed with these four general steps. Let’s dive in.
Let’s talk about data
Cleaning and preparing data for backtesting a trading strategy is the starting step in the process. It can be very challenging to find high-quality data (for a reasonable price), but unfortunately, it could be significantly more expensive not to get good-quality data. If you want to have accurate and reliable results, that is.
To clean data for backtesting, you should first ensure that the data is complete and accurate. This may require filling in missing data points or correcting errors. Next, you should remove any irrelevant data and ensure that the data is properly formatted.
The process of acquiring and cleaning data is very time-consuming and may require specialized skills and tools. As a result, traders usually simply purchase data from a data provider, expecting the data to be of high quality already. While this adds to the cost of backtesting, it also saves time and improves the quality of the results. Overall, the effort and cost involved in cleaning and preparing data for backtesting will depend on the specific needs and resources of the trader.
There are platforms that collect and clean the data for you and allow you to backtest your strategy easily. Such platforms are TradingView, MetaTrader, and cleo.finance.
The testing process
You have your data and you have your backtesting platform. Great! Now you can move on to the next step: the testing process itself. This involves using historical market data to simulate the performance of your strategy, and analyzing the results to see how well the tested strategy would have performed in the past. There are 4 major subcategories of backtesting:
- In-sample data
- Out-of-sample data
- Walk forward analysis
- Real-time testing = Paper Trading
Each of these steps provides additional incremental value that can help you reduce the potential for skewed results, due to biases that could’ve been introduced during the testing process.
“In-sample” vs “Out-of-sample” data
In-sample data refers to the historical data used to create and fine-tune a trading strategy. This data is used to build the model and develop the rules for making buy or sell decisions. The process of developing a trading strategy involves evaluating its performance on in-sample data, which helps in determining its accuracy, reliability, and robustness.
On the other hand, out-of-sample data refers to the data not used in the model development process. This data is used to evaluate the performance of the model and test its ability to generalize to new, unseen data. This evaluation is important because it helps in avoiding overfitting, which occurs when a model performs well on in-sample data but poorly on new data.
Out-of-sample data serves as a crucial test of the model’s ability to perform well in real-world conditions, and helps in avoiding optimistic bias, where a model is over-estimated based on its performance on in-sample data. Unfortunately, many traders tend to use only in-sample data to test and train their strategies. This leads to glorious backtest results, but horrendous live trading results. However, if your strategy performs well on out-of-sample data, it’s a good sign that it could be robust enough and not overfitted.
One way to use out-of-sample data is to test your strategy on a different time period than the one you used for in-sample data. For example, if you developed and optimized your strategy using data from 2016 and 2017, you could test it on data from 2018 to see how it would have performed in a different market environment.
Another way to use out-of-sample data is to test your strategy on different timeframes or asset classes. This can help you see how well your strategy would perform on data that is different from the one you used to develop and optimize it. By testing your strategy on different timeframes and asset classes, you can see if it’s robust enough to produce comparable results in different market conditions.
When testing your strategy with out-of-sample data, it’s important to watch out for outlier statistics. These are results that stand out as being significantly different from the rest of the data and from expectations. For example, if your strategy experiences a much larger drawdown than it did in the development process or has a different duration of trades when tested on out-of-sample data, it could be a sign that it’s overfitted and not robust enough.
Walk-forward analysis
Once you’ve tested your strategy with out-of-sample data, you can move on to the next step in the testing process: walk-forward analysis (WFA). This is a more advanced step that involves combining the in-sample and out-of-sample testing processes.
In a WFA, you develop and optimize your strategy on in-sample data, and then test it on out-of-sample data. Then, you move the data forward so that it encompasses the out-of-sample data from the previous period, and repeat the process of developing and optimizing your strategy on the new in-sample data, and testing it on the new out-of-sample data. This process is repeated until you have covered the entire time period you want to test with out-of-sample data.
The result of a WFA is a combined set of results from the out-of-sample data. This can give you a more comprehensive view of how your strategy would have performed in the past and can help you identify any potential issues or areas for improvement.
Paper Trading
Paper trading is the process of running your strategy in real-time, using demo funds instead of real capital. This is an important step because it allows you to see how your strategy would perform in live market conditions, without risking any of your own money.
However, paper trading has quite a vocal opposition from traders on social media. By paper trading your strategy, you can identify any issues or problems that you might not have noticed in the backtesting process, and make any necessary adjustments before going live. But one major thing paper trading can never help you with is determining how you would behave if you had been trading the strategy live.
The psychological aspect of being in a prolonged drawdown and losing money cannot be overstated. It will make you question your strategy (“maybe it doesn’t work”), question your strategy development process (“maybe I have introduced a cognitive bias by accident”), and you will try to tinker with it (“it would do great if this parameter was slightly looser”), while in reality all you might need to do is give it a bit of time.
Conclusion
The backtesting process is an essential part of developing a robust, trustworthy trading strategy you can lean on. By using in-sample, out-of-sample, and WFA data, you can see how your strategy would have performed in the past, and make any necessary adjustments to improve its performance. And by paper trading your strategy, you can see how it performs in real market conditions, and be prepared for the challenges and risks of live trading. By incorporating these steps into your strategy development process, you can increase your chances of success and maximize your profitability as a day trader.
If you want to try backtesting your strategy or develop a new one, you can do so for free on cleo.finance
