Backtesters: The Good, The Bad, And The Money

Grant Bartel
Mar 4, 2017 · 5 min read

Learn why backtesting trading strategies and portfolios is important and how to take backtest results with a grain of salt.

You wake up one day and think to yourself, “Woah, I have money.” Maybe you got it from saving, maybe from the lottery, or maybe someone just handed it to you. Either way, you now have some spare cash most likely sitting in a bank account doing nothing. Probably getting eaten up by inflation. No bueno.

You start to learn about how to invest your money to save for retirement, make a quick buck, or simply avoid the negative effects of inflation, and then

credit

you come up with an idea for a investment strategy! Then you think, “How can I test whether this strategy is any good?” Well unfortunately you can’t know for certain how well your strategy will do going forward without executing it, but you can certainly figure out how well it would have performed in the past, and that’s where a backtester comes in handy.

What is a Backtester?

A backtester runs a trading strategy — based on technical indicators or buy-and-hold — through historical market data to determine how well the strategy’s buying and selling of stocks, ETFs, or other assets would have performed in the past. This process gives us a viewpoint into the past and a better understanding of the effectiveness of our strategies.

While it’s true that past performance does not promise future returns, history often repeats itself and knowing the good and bad of any trading strategy requires an historical viewpoint. With that in mind, a backtester is an educational tool, not a crystal ball. While the results of a backtest might show that you’ve found something that’s going to make you rich, it’s likely that you’re guilty of overfitting (e.g., by tweaking and/or adding more parameters to a strategy to work perfectly over a specific time period).

But what makes a good backtester?

The Good

While a backtester demands attention to many subtle technical details, some extremely important characteristics include

  1. Accurately representing the past as it once occurred
  2. Quickly iterating through large amounts of historical time series data
  3. Running various types of trading strategies

Accuracy

Being able to execute a trading strategy as it would have been executed in real life while using data that represents true past events is arguably the most important characteristic of any backtester. Without an accurate representation of past events, any further analysis performed is of little importance and will cause a misunderstanding of the strategy. Garbage in, garbage out.

Good backtesters contain multiple features that deliver realistic, event-driven results, including using clean market data from reliable sources, accounting for bid-ask spreads when buying and selling assets, and executing trades at the market open prices based on the preceding market close prices.

Speed

No one wants to wait hours, even minutes for a backtest to run. Time is money. However, this can be difficult to avoid since computing potentially complex trading strategies against big data sets at scale requires a deep understanding of some of the most advanced technologies.

Good backtesters thrive in environments containing oceans of data and complex trading strategies. By leveraging key high-performance computing technologies, including cloud computing, vectorization, and highly optimized low-level code, trading strategies can be evaluated quickly at scale when using the right backtester.

Flexibility

There are literally an unlimited number of different trading strategies that exist in the world; from strategic asset allocation to technical indicator-based. Unfortunately, being forced to sit on either side of this spectrum, while also having to program, greatly reduces the number of possible trading strategies to be tested. The freedom to choose is imperative.

Good backtesters deliver on flexibility by running strategies based on strategic asset allocation, technical indicators, and anything in-between, without having to write a single line of code. By removing restrictive barriers for those willing to more actively participate in today’s financial markets, the evolution of a more flexible backtester must emerge as a solution to democratize the process of evaluating various types of trading strategies.

The Bad

It’s easy to abuse a backtester and fool yourself into thinking you’ll be the next Rockefeller. It’s important to realize the potential issues that could arise when using a backtester, especially overfitting.

Overfitting

Overfitting is arguably the biggest issue that comes up when performing a backtest mainly because you can trick yourself into thinking your strategy is an ATM machine but in reality it’s a money pit. No one wants to lose money, especially if it’s avoidable.

An example of overfitting could be as simple as someone following the process of

  1. Running a backtest on some strategy only to see poor results
  2. Adding more indicators and parameters to the existing strategy
  3. Rerunning the backtester on the strategy to find excellent results
  4. Executing the newly discovered strategy in real life and failing miserably

There are tools to help curb the potential of overfitting by testing for robustness (e.g., a strategy that works as designed in various market conditions), including scenario generators and walk forward analyzers. More on these in later posts.

The Money

So what were we doing again? Oh yeah, trying to make money.

credit

After coming up with some sort of investment strategy, generating results from a good backtester, and feeling confident you’ve avoided overfitting, it’s now time to see if you can reach your financial goal through live trading or paper trading.

While backtesting results cannot guarantee future returns, the generated insights can assist us in making more confident financial decisions. And in a world of uncertainty, especially within the financial markets, we can use all the help we can get.


We’re QuantSketch! We make it easy to quickly build, test, analyze, and monitor your trading strategies without writing a single line of code. Cool, huh? :)

QuantSketch

Quickly build and evaluate quantitative trading strategies without writing code.

Grant Bartel

Written by

Fintech Writer | Dev | Ambassador @ X-Team | 100% Remote. Check out my website to learn how I can help your Fintech business grow: https://grantbartel.com

QuantSketch

Quickly build and evaluate quantitative trading strategies without writing code.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade