Trading problems from physical perspective
Physical analogy
Search for a profitable trading strategy reminds me a prediction of a Brownian motion — the random motion of a particle in a fluid or a gas as a result of its collision with the ambient molecules.
Imagine kind of a Galileo’s experiment: grab a stone, stretch your arm, and let it drop. The stone will fall every time on the same spot on the ground right below your hand. The deviation in the trajectory of the stone is not noticeable by a human eye. Now take a feather, drop it. The trajectory of its fall is very different from drop to drop. The deviation of its landing spot is within tens of centimeters. Now imagine you drop a particle, say an oxygen molecule, and trace its motion from the same starting point. It will not fall, but will experience a nearly random motion. Why the behavior of the object is so different in these three cases? There is the force of gravity that pulls the object down, and there are repulsive forces due to collision with air molecules. The lighter the object is, the higher is the relative impact of the collision forces compared to the gravitational force.
The Brownian motion is unpredictable when we do not know the initial positions and the initial speed-vectors of all the particles in the system. But even if we knew them, non of the existent machines is capable to simulate a system with a large number of particles. We can only simulate a probability-cloud of the coordinates of the particle at certain time moment. Considering additional forces would increase the accuracy of the model. Gravity creates pressure gradient, and the higher the particle travels in the atmosphere, the larger it’s trajectory can deviate. A wind might blow the particles in certain direction. Increase in the ambient temperature will increase the speed of the particle and the distance it will travel. Pre-storm conditions might create an additional upward air drift. The motion of our particle becomes not 100% random. Carefully observing such physical phenomena and measuring their impact on the motion of the particle improves the accuracy of the prediction of its position.
Trading problem
Beginners in trading tend to dream about finding an indicator that will predict the price movement as precisely as the gravitational force affects the motion of a stone. Clear signal (gravitational force) on top of a barely visible noise (random collisions). More realistic goal is to find an indicator that reasonably predicts the motion of a feather, i. e. filters out the signal from the noise of the same level. The rapid expansion of the air surrounding a lightning strike (thunder), can be accurately simulated when the position and the time of the lightning is known. Such short-term movement is the analogy to the signals exploited in High Frequency Trading. For other common types of trading the analogy is predicting the direction of a nearly Brownian motion of an air molecule superimposed on a very weak wind with semi-permanent direction which needs to be established as precise as possible at every moment of time.
Here comes the key problem of finding such a reliable wind indicator in trading. In physics, it is relatively easy to proof with the scientific method that the predictive features are naturally re-occurring phenomena, which impact can be directly measured. In trading, there are hundreds of abstract indicators like Moving average, Bollinger Bands, Momentum etc, and millions of their combinations based on which we build our strategies. How do we know which indicators are reliable and which are not? We run the backtests on the historical data…
The problem is, on a limited-size historical data, one can always find a profitable strategy using a set of indicators with some logic to create buy and sell signals. It can be done with a naked eye. It can be ‘optimized’ using a brute force combination search. And real ‘masterpieces’ are created when one applies machine learning (ML). For example, using just 5 indicators and relatively simple ML algorithm we got 400% return per month on the historic data of bitcoin. Probably you already guessed that running this strategy on the unseen data showed highly-negative performance.
In the case of prediction the particle motion, one could pass the positions of the planets of the Solar system as parameters to train the ML algorithm, mistakenly assuming that the gravitational fields they create can visibly affect the joint particles drift. In one of the observations, the ML algorithm detects a good correlation of particles drift in the direction towards Venus in the morning, and towards Neptune in the afternoon, which was just a coincidence for that particular observation.
Only few of trading indicators might have a real cause — the wind on top of the Brownian motion. The examples are those related to inefficiency of the market mechanics or exploitation of the psychology of the of traders mass. The one who discovers the wind direction and exploits it before others — wins. Those who ‘discover’ useless indicators — lose.
Solutions
1. Historical data of the trading is the first and the most reliable source of information to build a strategy upon. At algologic.ai you can select, customize for you personal needs, and download the historical trading data from main crypto-exchanges. The data is available in ticker as well as candlestick formats, and includes the list of basic indicators.
2. Find re-occurring and explainable patterns/indicators. It is barely possible to fully apply scientific method here, but at least try to find a good explanation in terms of market-inefficiency or traders mass behavior. To explore the data you need good visualization tool for example Dataiku.
3. If you find few reliable indicators/patterns with a reasonable explanation and predictable power, you can apply machine learning strategies on top of them to further improve the performance.
4. Design and backtest your strategy based on your indicators. Use out-of-sample data to check if your patterns re-occur. In an article Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance, you can find more sophisticated mathematical tools to tackle this issue.
