Trading Strategy Optimization and Physics
Greetings, everyone! In several of our previous publications, we discussed in deep the technical details about the process of paper trading and backtesting a trading strategy in a high frequency context. In particular, we highlighted the order matching methodologies that were used in our market making application in the article “The Nitty-Gritty of Paper Trading a Market Making Strategy”. Suppose that you’ve carefully devised a strategy and performed thorough statistical testing on it and so far so good. Before launching it with one million real dollars, there is still one hidden dragon in the water. 🐲 This is the topic for today’s article. The discussions will be exploratory because we haven’t found a perfect solution to this problem yet. And the discussions will be quite entertaining if you happen to be interested in physics.
For all the past discussions in our published articles about paper trading and backtesting (as well as the majority from other people on the internet), there is an implicit assumption that our own trading activities will have zero effect on the market. In other words, our phantom orders are superimposed on top of the market like ghosts. However, we do know that the moment that we submit real orders to the real market, they will have an immediate observable consequence and other traders will react accordingly. This sounds somewhat similar to the so-called measurement problem in quantum mechanics in the field of theoretical physics: i.e. the moment that we perform a measurement on some physical system, its behaviors are inevitably changed. Linear superposition is only applicable in the ideal case. And for us, the ideal case is paper trading and backtesting. Once real trading starts and the capital that we deployed to the market isn’t trivial, things start to become complicated. Our own tradings can affect other people’s trading activities and their trading activities can then affect ours: which sounds like the so-called self-energy in quantum mechanics. Coming back to quantitative trading, we performed a comprehensive search on the internet and haven’t found any professional trading firm providing even the slightest hint about how they deal with such a problem of incorporating their own trade’s effects on the market into the simulation environment of paper trading and backtesting. So from a theoretical perspective we’ve created the following framework to handle it. Feel free to let us know if you disagree or have other alternative theories.
Let’s start with a thought experiment. We deploy certain amount of capital into the real market and record our order’s fills. At exactly the same time we run a paper trading program with the same amount of capital in the simulated environment and record our order’s fills. Because the simulated environment uses real market data and our own account’s conditions are the same, we’ll generate exactly the same set of orders at exactly the same time. The difference is that the real fills will deviate from the simulated fills. In normal cases the real fills should be smaller than the simulated fills due to competitions from other traders who essentially steal some fills from us. If we denote the ratio between the quantity of a simulated fill to the quantity of a real fill as r, then we can judge the realness of a paper trading methodology by looking at how close r is to 1: the closer the more real. In general r is a fairly unpredictable value depending on a plethora of factors because r is the results of the combined reactions from all other traders which is fairly unpredictable. Our theory described below can be used to obtain an approximate form for r and is applicable to whatever paper trading methodologies that you use, i.e. our theory’s goal is to introduce a factor which makes your paper trading process more real. We begin the theory by examining the asymptotic behavior that any such theory should reduce to in some extreme cases. On one hand, if our order’s quantity is infinitesimally small, the theory should reproduce the ideal scenario in which the effects of our own orders on the market are also infinitesimally small. This means that the simulated fill should be the same as the real fill, i.e. r should approach to 1 when our order’s quantity is infinitesimally small. On the other hand, if our order’s quantity is infinitely large, the theory should decay into another ideal scenario in which other trader’s orders are always infinitesimally small compared to ours and therefore our simulated orders should always end up being completed unfilled. This means that r should approach to 0 when our order’s quantity is infinitely large. One among many functions that has such properties is the exponential function:
r = exp(-k*orderQuantity) (1)
orderQuantity is our own order’s quantity and k is a constant characterizing the competition from other traders which causes our own order’s real filled quantity to be reduced. The procedure to determine k is as follows: We deploy certain amount of capital into the real market and record our order’s fills. At exactly the same time we run a paper trading program with the same amount of capital in the simulated environment and record our order’s fills. For each order, we know its quantity orderQuantity and we can calculate r by knowing its real fill and the simulated fill. Once we have executed a certain number of orders with some having large quantities, some having medium quantities, and some having small quantities, we can readily fit them into equation (1) and obtain the constant k. Done. From now on, whenever we have a simulated fill in the paper trading or backtesting mode, we multiply the filled quantity by a factor of r using equation (1) to make it more real.
If you are interested in our work or collaborating with us, join us on Discord: https://discord.gg/b5EKcp9s8T 🎉 We specialize in market data collection, high speed trading system, infrastructure optimization, and proprietary market making. You’re welcome to hire us as engineers, liquidity providers, traders, or asset managers.
Disclaimer: This is an educational article rather than investment/financial advice.