Custom “recursive” indicator in Python with backtrader

In the previous article (Custom Indicator Development in Python with backtrader) the development of a custom indicator was explore (the Stochastic indicator)

But some indicators have a particularity: they are recursive. This requires some extra management, because recursion needs to be solved for the calculation of the first value to be delivered (or else one remains stuck in the classic chicken-egg problem)

Let’s use the well known Exponential Moving Average (aka EMA) as an example of how a recursive indicator is developed. To simplify things, this is how the EMA is calculated

  • ema(0) = (1 — alpha) * ema(-1) + alpha * data

Where ema(0) is the current value and ema(-1) is the previous value (hence the recursion)

Although an exponential smoothing takes into account all the past values of a series, the trading industry (and traders) tends to think in terms of the period (or numbers of bars) to which the exponential smoothing is applied. The smoothing factor ( alpha ) can be calculated as follows

  • alpha = 2 / (period + 1

For those interested in further reading the article in the Wikipedia explains the details

The obvious impulse when creating the EMA in backtrader would be to do the following

import backtrader as btclass MyEMA(bt.ind.PeriodN):
lines = ('ema',)
params = (('period', 30),)
plotinfo = dict(subplot=False) # plot in same axis as the data
def __init__(self):
alph = 2.0 / (self.p.period + 1.0)
self.l.ema = (1 - alph) * self.l.ema(-1) + alph * self.data

Which creates the recursion conundrum because one needs the previous value to calculate the current one, and there is obviously no previous value at the beginning of the chain of calculations.

Note: the indicator subclasses PeriodN which already knows what to do with a parameter period to inform the platform about the minimum warm up period requirements

If one decided to run this code against the sample data in the backtrader distribution the graphical output would show the problem

The solution: add a seed value

next is the standard method for the step by step calculations in all objects with lines in backtrader, but there is one additional method which comes to the rescue for our problem of setting a seed value

  • nextstart which is called exactly once: when the minimum warmup period for the indicator is met. It defaults to delegating the work to next but in this case it will seed the calculation
import backtrader as btclass MyEMA(bt.ind.PeriodN):
lines = ('ema',)
params = (('period', 30),)
plotinfo = dict(subplot=False) # plot in same axis as the data
def __init__(self):
alph = 2.0 / (self.p.period + 1.0)
self.l.ema = (1 - alph) * self.l.ema(-1) + alph * self.data

def nextstart(self):
period = self.p.period # for readability
self.l.ema[0] = self.data.get(size=period) / period

But if we were to run this in the default execution mode backtrader uses (runonce=True) this would simple deliver no value and the chart, just like above, would also be empty.

Running it with cerebro.run(runonce=False) does the magic as seen in this chart.

Why this?

  • When running in the default mode, backtrader calculates things in vectorized mode and this has a consequence: self.l.ema(-1) is a made to be a vector of values (i.e.: an array) which is precalculated before being using for the calculation of self.l.ema

Which simply means that the declarative approach (we declared the calculation method during __init__ , hence the declarative nature) combined with recursion faces restrictions when working in vectorized mode.

Can it be done?

Yes it can. But in this case one can forego the declarative approach and perform all calculations manually using the dirtiest inner tricks known only to advanced *bactraders*

import backtrader as btclass MyEMA(bt.ind.PeriodN):
lines = ('ema',)
params = (('period', 30),)
plotinfo = dict(subplot=False) # plot in same axis as the data
def oncestart(self, start, end):
# Calculate a seed value for the EMA
src, dst = self.data.array, self.line.array
period = self.p.period # for readability
dst[start] = sum(src[start - period + 1:end]) / period
def once(self, start, end):
src, dst = self.data.array, self.line.array
alpha = 2.0 / (self.p.period + 1.0)
alpha1 = 1.0 - alpha
ema1 = dst[start - 1]
for i in range(start, end):
dst[i] = ema1 = alpha1 * ema1 + src[i] * alpha

In this case the methods oncestart and once have been used (recall the parameter for cerebro is calledrunonce and this allows the calculation of the indicator in the faster runonce=True mode in backtrader without being worried about recursion problems.

Of course, one doesn’t usually write two different versions of an Indicator, but rather one. See the source code for the ExponentialSmoothing indicator which is used in backtrader (the ExponentialMovingAverage is a wrapper to make it part of the larger family of moving averages) See it here

Written by

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store