Lessons learned on designing crypto trading strategies

Many traders dream about writing their own bots that will trade autonomously while they are away from their trading desk. We have learned that a proper backtesting is the way to make this dream to come true. In this article, we will describe a few points that we consider essential on backtesting a trading strategy, and our lessons learned.

Backtesting is the process of validating whether a trading strategy is profitable. Basically, we simulate the strategy using the historical data. If the results are positive, there is a good chance the strategy may work in the real trades. However, this is not always the case. A live trading environment is a dynamic system that cannot be predicted beforehand. For example, in a few cases, we observed that placing an order into the order book changes the direction of the market — this cannot be anticipated or simulated during the backtesting. Taking into account such limitations, we consider the backtesting as a necessary but not sufficient step in the strategy-development life-cycle. Although limited, ignoring the backtesting or its results is a receipt for disaster in algorithm trading.

There are a few key points we pay attention to when backtesting. This does not guarantee our strategy will work but it has helped us to validate our assumptions so far. These steps are:

1. The strategy design
2. The backtesting metrics
3. What data to use
4. Implementation
5. Results visualization and validation
6. Discarding or deploying the strategy

The strategy design

The first step is to choose a well defined strategy to backtesting. Simpler strategies are easier to backtest, so we start with simpler assumptions first. For this exercise, let’s consider we will use 1 minute candlestick historical data. Suppose our strategy definition is:

We buy when the high price is below the 20 moving average, and we sell when the price is 1% above or below the buying price.

Note that it is very important to correctly state the buy and sell moments. In the definition above, we did not specify precisely when to buy. We still would have the questions: should we buy using the market price or place a limit order at a fixed price? In the latter case, for how long we leave the order open and what would be the buying price? What volume to buy? All these should be answered in the strategy definition. Therefore, a better definition would be:

We buy $10K by the market price when the high price is below the 20 moving average, and we sell when the price is 1% above or below the buying price.

Notice that this is a fictional strategy and we do not recommend to use it.

In a nutshell, we always backtest the volume we want to buy, the buying and selling price, and also the type of order we want to place. Backtesting limit orders requires more attention as we need to simulate the behavior of limit orders as well.

The backtesting metrics

Once the strategy is determined, we need to define how to measure its performance. There are several metrics we used: sharpe ratio, max drawdown, profit in equity, profit in percent, average profit per trade, number of trades, average time per trade, among others. Note that different strategies may require different metrics. Check the literature online, there are plenty of measures that may suits better to your needs.

For the strategies we designed at algologic.ai, apart from all the metrics above, we always looked into the worse trade. It is easy to come up with strategies that generate profit over 99% of the trades, but then there is 1% of the trades that eats all the profit. Understanding the metrics requires a lot of time and trial and error, but saves us time on mistakenly deploying unprofitable strategies.

What data to use

We use different data for different strategies. Mainly we use candlestick data or tick data. For more advance strategies, we use the order book, social media or news, as well. Our reliable datasets saves us a lot of headache during backtesting. Gaps on data can create an illusion that the strategy is working, and it makes things harder to debug. Unfortunately, many datasets online contain gaps. At algologic.ai, we have built an entire system to collect and validate our data. All the data used are checked via systematic data quality routines and manual inspection. Please visit our webshop in case you need high quality data for backtesting. Using our data will save time on data collection, processing and cleaning.

Implementation

Several tools exist for backtesting trading strategies. At algologic.ai, we have built our own backtesting system because we need to work with tick data at the finest precision and incorporate advanced machine learning in the backtesting. Designing a proper backtesting system is a long and time consuming project. It is a complex endeavor that should not be underestimated. Our backtesting system can simulate properly buy/sell volume, as well as limit and market orders. Importantly, we have also designed a tool to plot the trades generated by the backtests. The ability to visualize the trades during the backtesting is essential to improve our strategies.

Results visualization and validation

The last step in the backtesting is to visualize the results of the trades. It is important to see the exact time the backtest places the buy and sell orders, and their actual execution time on the candlestick or price charts. This is mechanism for us to access whether the backtesting is correct and whether it does the job we expect it to do. This process is the most straightforward way to spot if the strategy implementation is correct. In case you need a visualization tool for this purpose, please contact us, we can license it under request.

Discarding or deploying the strategy

Once we inspect the trades and confirm the strategy is simulated correctly in the backtesting, it is time to check the metrics and to see whether the strategy is potentially profitable or not. It was common that the majority of the strategies we validated were not profitable. We notice that it is important to look into all the metrics, and verify whether all of them give a good indication about the strategy performance. As mentioned before, high accuracy on some metrics may hide the bad trades. We always look into the worse trade and analyze its impact overall. It is a good indication that the strategy, although profitable, can lead into big loses. Also, backtesting using a several different windows of time is more precise and effective than backtesting in a short period or long period of time. We carefully select the market condition that we want to backtest. Usually, we select periods of extreme volatility and low volatility as the corner cases to validate. Out of 100 basic strategies we validated, only two showed to be profitable and reliable. We have moved them to live trading and it shows a very close return to our backtesting. It is clear that the systematic procedure we briefly described here is effective and reliable for us.

Conclusion

Strategy backtesting is a hard work, necessary but not enough. To be done correctly, it requires a lot of prior trading knowledge and expertise on software engineering. Our adventure on building our own system showed us that this is a discipline that require long time to master and should not be underestimated by novices.

In this article we described our experience on backtesting our trading strategies, and the elements we consider relevant to share. Hope our lessons learned will be useful for the new quantitative and algorithm traders that plan to run their own backtesting system.