Backtest MasterClass — Part 1

Published in

Mudrex

13 min readJun 23, 2020

Understanding the Backtest Report

If you knew your trading strategy would give correct signals only 50% of the time, would you commit your scarce savings to trade with it? Now, what if it worked 75% of the time? Would you still be hesitant? Backtesting gives you the confidence to know that your trading strategy will work. Backtesting is simply the intentional testing procedure to verify that your trading strategy works using historical (back) market data and market conditions.

Gone are the days when backtests were expensive and could only be performed by institutional investors or money managers. Mudrex can help you run and analyse a backtest for your crypto-auto trading bot within seconds!

See it for yourself here!

In the previous MasterClasses, we’ve talked about some very well known technical indicators ( RSI, MACD, Volume Indicators etc.) and how to use them to make your own auto-trading strategy. However, the key to making a good trading strategy isn’t only humongous cumulative returns in your backtest. Risk management plays an important role too. This is a MasterClass on how to understand, analyze and improve upon every single metric of the backtest results which will help you improve your strategy to give out the best returns at minimum risk. We’ll be covering the following metrics and risk management techniques in detail in the following articles:

Part 1: Understanding the Backtest Report
Part 2: Getting better at Sharpe Ratio
Part 3: Understanding Maximum Drawdown
Part 4: Stop-Loss and Take-Profit
Part 5: Understanding the Order Book

In this article, we’re gonna talk about understanding all twenty-two backtest metrics offered by Mudrex viz.

Monthly avg returns
Loss making months
Max drawdown
Total trades
Sharpe ratio
Profit factor
Overall returns
Market returns
Winning streak
Losing streak
Trades won
Largest winning trade
Largest losing trade
Gross profit
Gross loss
Avg profit on win
Avg loss on lost
Average exposure time
Long performance
Short performance
Total fees
Max realized drawdown

Snapshot of a backtest results screen on Mudrex

Introduction

Backtesting is the process of applying a trading strategy or analytical method to historical data to see how accurately the strategy or method would have predicted actual results. It is solely based on the assumption that “History repeats itself” and relies on finding profitable patterns or trends in the historical market price data which can be anticipated in the future for similar market conditions.

Generally, traders believe that the prices in cryptocurrency markets aren’t usually affected by external factors like news, trade wars, quarter results, government policies etc. Cryptocurrency prices are generally affected by mere supply and demand and thus backtesting a strategy on prior price patterns if analysed religiously, can help predict future patterns to a profitable extent, to say the least.

Hence if you’re proposing that shorter moving averages crossing up longer moving averages might be a bullish sign, you can simply test this strategy on historical price data and see the results for yourself. If the strategy is working for the majority of the trades and giving out decent positive returns without much risk involved, you might want to implement it on live markets and generate profits for yourself. Otherwise, improvise, adapt and overcome by iterative analysis of the following metrics.

Monthly Average Returns

Monthly Avg. Returns is a simple average of total cumulative returns of the strategy. Returns are calculated as a percentage of the overall invested amount on Mudrex. Avg. Returns are more important than the Overall Returns because a strategy might be backtested on data as long back in time as 3 years. Thus it’s important to know the returns averaged every month so that you can expect similar returns for the duration you’re willing to make your strategy go live.

Mere positive returns do not make your strategy worth going live and can cost your negative returns in the times to come. Positive returns is a necessary but insufficient condition for a strategy to be called good. It is important to know the conditions of markets (choppy or trending) in which the strategy performed better than the rest of the period. The strategy can then yield positive returns if similar market conditions are anticipated.

Loss Making Months

Loss making months, as the name suggests, is the percentage of months where the monthly returns were negative. If the loss making months are greater than 50%, it means you might have to incur losses for the majority of your trading period. But the overall performance of the strategy also depends on the volume of these negative returns.

Maximum Drawdown

A maximum drawdown (MDD) is the maximum observed loss from a peak to a trough of a portfolio, before a new peak is attained. Maximum drawdown is an indicator of downside risk over a specified time period. It is a metric which represents the losses in the “worst possible trade” during your backtest period.

Maximum drawdown (MDD) is an indicator used to assess the relative riskiness of one stock screening strategy versus another, as it focuses on capital preservation, which is a key concern for most investors.A low maximum drawdown is preferred as this indicates that losses from investment were small. If an investment never lost a penny, the maximum drawdown would be zero. The worst possible maximum drawdown would be 100%, meaning the investment is completely worthless.

More on Maximum Drawdown and Realised Maximum Drawdown will be taken up during the Part — 3 of this MasterClass.

Total Trades

Total trades is simply the total number of trades the strategy managed to execute during the backtest period. A very high number of total trades does not make sense because it means that the conditions of the strategy are getting executed very at high frequencies and a lot of these signals might be false, reducing the overall performance of the strategy. Trades can be visualized on the candlestick chart by going to the “Pro Graph” section of backtest results. If the number of total trades is high, and the buy/sell signals are too close to each other in the “Pro Graph” it means that the buy and sell conditions are very close mathematically and as soon as a buy condition is fulfilled, the sell condition is close too thus not leading to great profits generally. This can also be analysed using “Average Exposure Time” which we’ll talk about a little later.

Sharpe Ratio

Sharpe ratio indicates traders’ desire to earn returns which are higher than those provided by risk-free instruments like fixed deposits with guaranteed returns without any downside. Sharpe ratio is based on standard deviation which in turn is a measure of total risk inherent in a trading strategy, Sharpe ratio indicates the degree of returns generated by a strategy after taking into account all kinds of risks. It is the most useful ratio to determine the performance of a strategy.

The higher the Sharpe ratio of a portfolio, the better is its risk-adjusted-performance. However, if you obtain a negative Sharpe ratio, then it means that you would be better off investing in a risk-free asset than the one in which you are invested right now. Sharpe Ratio benchmarks as such that-

Negative: Worse
Less than 1: Bad
1 to 2: Adequate/Good
2 to 3: Very Good
Greater than 3: Excellent

It’s all about maximizing returns and reducing volatility. If an investment had an annual return of only 10% but had zero volatility, it would have an infinite (or undefined) Sharpe Ratio. Of course, it’s impossible to have zero volatility, even with a government bond (prices go up and down). As volatility increases, the expected return has to go up significantly to compensate for that additional risk. The Sharpe ratio reveals the average investment return, minus the risk-free rate of return, divided by the standard deviation of returns for the investment. Below is a summary of the exponential relationship between the volatility of returns and the Sharpe Ratio.

Profit Factor

The profit factor is defined as the gross profit divided by the gross loss (including commissions) for the entire trading period. This performance metric relates the amount of profit per unit of risk, with values greater than one indicating a profitable system. We all know that not every trade will be a winner and that we will have to sustain losses. The profit factor metric helps traders analyze the degree to which wins are greater than losses.

Overall Returns

Overall returns, as the name suggested, are the total cumulative % returns over the principal amount invested. It represents the total money you have gained/lost over the period of backtest as a percentage of the amount invested. However, you might want to judge your strategy returns based on monthly avg. returns over overall returns and set your expectations accordingly because you might not make your strategy go live for the same time as that of your backtest. For instance, if your strategy is yielding 200% returns in a 2 year backtest, it does not mean that you can get the similar returns if you’re planning to make your strategy go live for 6 months. Returns will come down proportionally (if at all your strategy is good enough to give similar monthly returns)

Market Returns

Market returns are basically the returns you could’ve expected by simply investing money in a currency without a trading strategy. It is the % difference of the market price at the start of the backtest from the market price at the end of the backtest. Thus if the price of Bitcoin has increased ever since the beginning, you’ll be able to see high market returns, probably even higher than your overall returns. Hence it is important to compare the Overall Returns and the Market Returns.

The purpose of implementing a trading strategy is to buy low and sell high as efficiently as possible. A strategy should therefore be expected to give out Overall Returns more than the Market Returns. If your strategy is giving out returns less than the market returns, you might even have bought at first open and held till last close to give out returns better than that of your strategy. Hence it is important to have a positive Improvement over the Market.

Winning/Losing Streak

It is simply a return metric which gives you the maximum number of consecutive winning/losing trades respectively. A winning streak greater than a losing streak means your strategy is more likely to give out positive returns for a long period, without any losing trades in between. It is used for the general understanding of individual trades without having to look at the trade log.

Trades Won

Trades won is basically the percentage of the number of winning trades out of total trades. It is not a complement to Loss Making Months because it is based on the number of trades and not on monthly returns of the strategy. It is also different from Profit Factor as Profit Factor is based on gross profits whereas Trades Won is about the number of winning trades not the win value.

A lot of traders believe that you only ought to be right 51% of the time. If you’re sure as hell your strategy will give out positive returns more than half the time, your strategy might be worth implementing. Therefore, a Trades Won percentage of greater than 50% is to be kept as a benchmark for a good strategy.

Largest Winning/Losing Trade

This is simply the maximum profit/loss incurred in a trade as a percentage of the principal amount invested. Generally, Largest Winning Trade is expected to be greater by a good margin than Largest Losing Trade for a good strategy.

Gross Profit/Loss

Gross Profit/Loss is simply the sum of profits/losses incurred as a percentage of principal invested amount. When Gross Profit is greater than Gross Loss, Overall Returns are positive. Gross Profit is always positive whereas Gross Loss is always negative.

Avg Profit/Loss on Win/Lost

Avg. Profit on Win is basically the average profit one can expect on a winning trade as a percentage of the amount invested. The greater this value is, the more profitable the strategy can be. Similarly, Avg. Loss on Lost is the average expected loss on a losing trade.

Average Exposure Time (AET)

This is a very important metric to analyse the average period of time between the opening and closing of a trade. It is the average time difference between the generation of buy signal and the generation of the corresponding sell signal. It tells us about the average time for which the currency is held in a trade.

A very small AET would mean that the buy/sell conditions aren’t very different and thus execution of buy signal means a sell signal is also near and thus the currency is not held for long enough to generate good profits. An AET too long means that the local tops and bottoms aren’t being used efficiently and thus your Overall Returns will be similar to Market Returns and thus the strategy isn’t really working.

AET also depends on your frequency of trading strategy. If you’re trying to develop an intraday trading strategy, your AET will be in seconds, minutes or hours. if you’re making a day trading strategy, your AET will be in units of days. A swing trading AET will be in units of days or even weeks, depending on the frequency of trading strategy.

Long/Short Performance

Long Performance is the portion of Overall Returns that comes from all the long trades i.e. buying small selling large. Short Performance is the portion of Overall Returns that comes from all the short trades i.e. selling large buying small. This is a very interesting metric to utilise when you’re trying to make a Long-Short strategy and want to know the individual performance of all the long trades and short trades. If either of Long or Short Performance is negative, it is best to use the strategy in Short Only or Long Only mode respectively.

Total Fees

Total Fees is the amount paid to the exchange as a brokerage for your trades. With every trade, you need to pay a small percentage of trade amount to the exchange as a service tax. The percentage of this fee is different for different exchanges and brokers. People generally use exchanges with minimal trading fees. You can set your own percentage depending upon your exchange/broker while performing the backtest.

Setting the Fee Percent depending on your Exchange

Summarising a Decent Trading Strategy

Although the expectations from a backtest is subjective to different traders and their trading styles, one can expect the following points at minimum to see if his/her strategy is working or not:

Positive Monthly Average Returns (obviously :P)
Less than 50% Loss making months
Minimum Max Drawdown (-25% for most traders)
Sharpe Ratio > 1
Maximum Profit Factor (Greater than 1)
Overall Returns > Market Returns
Winning Streak > Losing Streak
More than 50% Trades Won
Largest Winning Trade > Largest Losing Trade
Avg Profit on Win > Avg Loss on Lost
Positive Long as well as Short Performance

Backtest Results of a decent Trading Strategy on Mudrex

However, even if your trading strategy satisfies all of the above conditions, it still might not be worth deploying into the live markets. Your results need to be exceptional and way better than the above to be able to risk your money with it. It requires a lot of experience and hard work with hundreds of backtests and strategy iterations to get to a deployable trading strategy. Feel free to head up to our platform and test innumerable auto-trading strategies for free without having to code a single line!

For backtesting to provide meaningful results, traders must develop their strategies and test them in good faith, avoiding bias as much as possible. That means the strategy should be developed without relying on the data used in backtesting. That’s harder than it seems. Traders generally build strategies based on historical data. They must be strict about testing with different data sets from those they train their models on. Otherwise, the backtest will produce glowing results that mean nothing.

One way to compensate for the tendency to data dredge or cherry pick is to use a strategy that succeeds in the relevant, or in-sample, time period and backtest it with data from a different, or out-of-sample, time period. If in-sample and out-of-sample backtests yield similar results, then they are likely generally valid.

This means that one needs to ensure testing the strategy over multiple tick-intervals and backtest periods to prevent overfitting of the strategy to historical data. In the upcoming article of our MasterClass on Backtest, we will be talking about how to improve upon one of the most important metrics of the backtest results, the Sharpe Ratio.

Stay tuned!