SUPERALGOS DATA MINING
Highlight Market Singularities to Differentiate Anomalies and Trend Reversal
What if we could guess the next market state with a quick glance at the current price?
In this article we propose a method derived from the reduced normal law to identify the probability the price had to occur and highlight anomalies in the periodic return. Using the same principle, we will calculate price probability bands to statistically identify what is the most probable level the price can reach.
Statistical Studies of the Price, the Return and Realized Probability
Considerations on the statistical distribution of price and periodic return
To a certain extent, the price of an asset can be considered as normally distributed around the moving average, i.e. it is a random variable. The distribution is a continuous probability distribution and is described by the Laplace-Gauss probability density:
With σ the standard deviation, and μ the mean.
The normal distribution of the price is actually an approximation since the price is more accurately described by the Lévy distribution. Considering the calculation effort implied by the Lévy distribution the normal distribution is a fairly good approximation and has proved is usefulness in the domain as shown by the indicators using volatility as a calculation basis.
If we use the moving average and the moving standard deviation as in the so-called Bollinger Bands, we can consider the probability density function describes the distribution over the period used to calculate the moving average.
The reduced normal distribution is the distribution of a random variable where the mean value is null and standard deviation is 1.
Using reduced normal law to determine the realized probability of the price
The reduced normal law is normalized with null mean value and standard deviation equal to 1. As is, the use-cases are limited. In order to use it to deduce the realized probability of the price we have to normalize the price. It is achieved with the Z normalization:
Basically, Z normalization describes the position of the price compared to the mean, expressed in units of the standard deviation. Z can be either negative or positive, respectively depending on the position of the price below or above the mean.
Thus, the probability can be deduced from an abacus table of the normal reduced law:
To use the table, we simply have to decompose Z as the unit and first digit on the left column and second digit in the horizontal headers. The resulting number then give the probability associated to the value of Z.
An increasing Z absolute value shows the variable x (the price) is moving away from the mean value. Consequently, the farther x is from the mean, the lowest the probability of this event to occur.
A very well-known indicator is taking advantage of this density of probability: the Bollinger Bands. Bollinger Bands are calculated as a center line, the moving average of the typical price HLC3 (max + min + close)/3 of a period and an upper and a lower band respectively at twice the standard deviation from the center line. If we normalize the price to cast the Bollinger Bands on the reduced normal law, we observe the upper and lower bands have Z = 2.0, and a corresponding probability of 2.3% chances to occur upon the time frame of the number of periods of the moving average.
The above graph shows an example of Z-normalization with Z=2 (orange) and Z = 2.34 (red), respectively for 2.3% (Bollinger Bands) and 1% probability with BTC/USDT closing price at the 1-hour time frame based on the 20 periods moving average. The black curve oscillates around 0, describing an upward trend while Z > 0 and a downward trend while Z < 0. A sharp variation of price is characterized by a peak on Z with values significantly above or below the 1% probability level. Looking at the point where Z crosses the orange line, we can guess the Bollinger bands type behavior, with price bouncing from lower band to upper band.
Distinguish a price anomaly from a trend reversal with the periodic return
Among the forest of spikes given by the Z normalization of price, some are related to a change in the trend and others seems to be just anomalies and the price recovers in a few periods, keeping the pace of the trend. The last ones are actually anomalies in the periodic return, the ratio between the actual price ant the previous period closing price:
Using the same method than for the price, we can proceed to the Z normalization of the Periodic Return.
The above graph shows the Z normalization of the periodic return calculated with 200 periods moving average on the 1-hour chart of BTC/USDT pair and at the same period than for the previous graph of Z price.
Plotting the number of occurrences within a certain probability range, we observe 80% of the values are in the 15% chance probability range and 95% within 2 time the standard deviation. Value as low as Z = 12.4 can be found (1 occurrence on more than 16 thousand). The price curve effectively shows a sharp price drop or increase once Z drops below or above low probability limit with most of the time a sharp recovery.
Superposing the Z normalization of the periodic return and price shows clearly corresponding peaks as non-corresponding peaks in terms of probability level. It seems actually corresponding peaks at low probability are more likely to not tend to a price recovery as singular peak of the periodic return normalization will lead to a recovery to the current trend.
Realized price’s probability and return
Looking at the Z normalization, if we calculate the standard deviation and the moving average, we can calculate the price correponding to a certain probability it has to occur, then chosing a probability value in the reduced normal law table to fix Z value :
For the price at a level of probability above the moving average, and :
For the price at a level of probability below the moving average.
We can easily chose different probability to trace the levels on the price char.
The above chart shows probability levels at 0.2%, 1%, 2,3% (Bollinger bands) and 30% around the 20-periods moving average with the closing price at 1-hour timeframe for BTC/USDT pair.
The patterns at the chart reveal intersting behaviors of the price. If we clearly see the Bollinger bands bouncing, the 30% probability level actually acts as a dynamic resistance level after the rebound on the lower 2.3% probability band. If the price does not cross the lower 30% level, it goes down, oscillating and bouncing below this level. On the other hand, once the lower 30% level is crossed, the upper 30% level then becomes the resistance level, and once crossed turns into the support level of the upward trending price. In case the upper 30% level is not crossed, the lower 30% level becomes the support level. Then, if the price bounces between the upper and lower 30% level, we observe a squeezing of the probability bands : the volatility decreases and it corresponds to the intermediate moment where the Keltner channels frame the Bollinger bands.
Now the same chart can be done with the periodic return.
We plot the measured Periodic Return with probability levels at 0.01%, 0.02%, 1%, 15% and 30%. The biggest proportion of periodic return is contained within the 30% probability band. We observe each time the periodic return hit the 1% level (or cross under) it goes up to the 30% level above the mean periodic return. It is a sign of price stabilization after a sharp movement and could be possible good entry points for a position.
Integration of the Realized Price and Periodic Return Probabilities, and Probability Levels in Superalgos
Integrating indicators with the price and return probabilities is quite simple. The JavaScript code for each is no more than 50 lines.
Realized price probability
The realized price probability is the Z normalization of the price. To calculate it, we fisrt need to fetch the candles on 20 periods and then proceed to the moving average and standard deviation calculation. Then we can calculate Z :
let candle = record.current
let len = 20 //periods for MA calculation//fetching last len typical price
variable.TP = (candle.max + candle.min + candle.close) / 3
variable.last20TP.push(variable.TP)if (variable.last20TP.length > len) {
variable.last20TP.splice(0, 1)
}
variable.sTP = 0
variable.sTPSquare = 0
for (var i = 0; i < len; i++) {
variable.sTP += variable.last20TP[i]
}//calculate MA
f (variable.last20TP.length === 20) {
variable.MA = variable.sTP / variable.last20TP.length
variable.sigma = Math.sqrt(variable.sTPSquare /
variable.last20TP.length)
} else {
variable.MA = variable.TP
}//calculate standard deviation
if (variable.last20TP.length === 20) {
for (var i = 0; i < len; i++) {
variable.sTPSquare += Math.pow(variable.last20TP[i] - variable.MA, 2)
}
variable.sigma = Math.sqrt(variable.sTPSquare / variable.last20TP.length)
} else {
variable.sigma = variable.MA * 0.01
}
variable.Z = (candle.close - variable.MA) / variable.sigma
Realized return probability
Based on the same principle, the calculation of the Return probability uses the 200-periods SMA of the Periodic return. In a first time, we need to calculate the periodic return and store it in a 200 slots array. We then calculate the SMA and the standard deviation. Finally we proceed to Z-normalization of the periodic return.
let candle = record.current
let len = 200 //periods//calculate and store Periodic return
if (variable.prevClose === undefined) {
variable.prevClose = candle.max
}variable.PR = Math.log(candle.close / variable.prevClose)variable.lastPR.push(variable.PR)if (variable.lastPR.length > len) {
variable.lastPR.splice(0, 1)
}variable.sTP = 0variable.sTPSquare = 0for (var i = 0; i < len; i++) {
variable.sTP += variable.lastPR[i]
}//calculate MA
if (variable.lastPR.length === len) {
variable.MA = variable.sTP / len
variable.sigma = Math.sqrt(variable.sTPSquare / len)
} else {
variable.MA = variable.PR
}//calculate standard deviation
if (variable.lastPR.length === len) {
for (var i = 0; i < len; i++){
variable.sTPSquare += Math.pow(variable.lastPR[i] - variable.MA, 2)
}
variable.sigma = Math.sqrt(variable.sTPSquare / len)
} else {
variable.sigma = variable.MA * 1 / 100
}variable.Z = (variable.PR - variable.MA) / variable.sigmavariable.prevClose = candle.close
Price probability levels
The calculation of the probability levels use the inverted Z-normalization where we sear for the price. As per the other indicators of thi set, we have to evaluate the moving average, on 20 periods, and the standard deviation. We then use the reduced normal law table to fix Z at the desired probability level.
let candle = record.current
let len = 20 //periods for EMA calculation//fetching last len typical price
variable.TP = (candle.max + candle.min + candle.close) / 3
variable.last20TP.push(variable.TP)
if (variable.last20TP.length > len) {
variable.last20TP.splice(0, 1)
}variable.sTP = 0
variable.sTPSquare = 0for (var i = 0; i < len; i++) {
variable.sTP += variable.last20TP[i]
}//calculate MA
if (variable.last20TP.length === 20) {
variable.MA = variable.sTP / variable.last20TP.length
variable.sigma = Math.sqrt(variable.sTPSquare / variable.last20TP.length)
} else {
variable.MA = variable.TP
}//calculate standard deviation
if (variable.last20TP.length === 20) {
for (var i = 0; i < len; i++) {
variable.sTPSquare += Math.pow(variable.last20TP[i] - variable.MA, 2)
}
variable.sigma = Math.sqrt(variable.sTPSquare / variable.last20TP.length)
} else {
variable.sigma = variable.MA * 0.01
}variable.LP1 = variable.MA - variable.sigma * 2.34variable.LP30 = variable.MA - variable.sigma * 0.53variable.HP1 = variable.MA + variable.sigma * 2.34variable.HP30 = variable.MA + variable.sigma * 0.53variable.LP02 = variable.MA - variable.sigma * 2.95variable.HP02 = variable.MA + variable.sigma * 2.95variable.LP01 = variable.MA - variable.sigma * 4.87variable.HP01 = variable.MA + variable.sigma * 4.87
Return probability levels
The return probability levels use the same principle as the price probability levels but with the periodic return as a calculation basis and a 200-periods moving average.
let candle = record.current
let len = 200 //periods//calculate and store Periodic return
if (variable.prevClose === undefined) {
variable.prevClose = candle.max
}
variable.PR = Math.log(candle.close / variable.prevClose)
variable.lastPR.push(variable.PR)if (variable.lastPR.length > len) {
variable.lastPR.splice(0, 1)
}variable.sTP = 0
variable.sTPSquare = 0for (var i = 0; i < len; i++) {
variable.sTP += variable.lastPR[i]
}//calculate MA
if (variable.lastPR.length === len) {
variable.MA = variable.sTP / len
variable.sigma = Math.sqrt(variable.sTPSquare / len)
} else {
variable.MA = variable.PR
}//calculate standard deviation
if (variable.lastPR.length === len) {
for (var i = 0; i < len; i++) {
variable.sTPSquare += Math.pow(variable.lastPR[i] - variable.MA, 2)
}
variable.sigma = Math.sqrt(variable.sTPSquare / len)
} else {
variable.sigma = variable.MA * 1 / 100
}variable.LP1 = variable.MA - variable.sigma * 2.34variable.LP30 = variable.MA - variable.sigma * 0.53variable.HP1 = variable.MA + variable.sigma * 2.34variable.HP30 = variable.MA + variable.sigma * 0.53variable.LP02 = variable.MA - variable.sigma * 2.95variable.HP02 = variable.MA + variable.sigma * 2.95variable.LP01 = variable.MA - variable.sigma * 4.87variable.HP01 = variable.MA + variable.sigma * 4.87variable.LP15 = variable.MA - variable.sigma * 1.03variable.HP15 = variable.MA + variable.sigma * 1.03variable.prevClose = candle.close
Conclusion
Along this article, we have shown how to use reduced normal law to build a set of indicators higlighting the realiyed probability and te probability levels of the price and the periodic returns. We observe sharp price movement leads to a singularity in the probabilities, showing the low probabilities and event has to occur. In the mean time we see those singularities lead to a reaction in the price movement. While builing the probability levels, we have shown those levels could be used as dynamic support and resistance level while trading the trade with indicators like the Bollinger Bands.
This new set of indicators is available at the Polus Datamine in Superalgos with at least ten probability levels usable. Number of periods for the base moving average can be easily adapted as the probability levels using the table provided in this article
All the material presented here can be reused and integrated freely on the condition linking to this article and the Superalgos website.
Disclaimer: The content of this article is for educational purpose only and does not constitute financial advice. Trading is not suitable for everybody; seek professional advice. Use this article at your own risk.
If you enjoyed this article and want to participate in the most promising open-source social trading platform… come and join us !
- Superalgos Community : https://t.me/superalgoscommunity
- Superalgos Support Group : https://t.me/superalgossupport
- Superalgos Develop Group: https://t.me/superalgosdevelop
- Superalgos Data Mining : https://t.me/superalgosdatamining
- Superalgos Machine Learning Group: https://t.me/superalgosmachinelearning
- On Discord Server: https://discord.gg/CGeKC6WQQb
- Superalgos en Français : https://t.me/superalgos_fr
