Introduction
Quantitative investing can be Simple, Easy, Awesome.
For quantitative researchers, their day to day responsibilities are just one of the three following tasks: Security Selection, Portfolio and Risk Management, and Automatic Trade Execution.
Backtesting, according to Investopedia, is the general method for seeing how well a strategy or model would have done ex-post. Backtesting involves in every stage of the quantitative research process — from security selection, portfolio construction to automated trading. Backtesting normally is conducted with historical data, but I believe it actually can be more valuable with the simulated data, another term a.k.a Scenario Analysis or Stress Test.
Data Collection and Data Wrangling
Let me have a word for friends just starting to study quantitative finance: in order to get free pricing data, the easiest way is from Yahoo! Finance:
# download prices from yahoo financefrom yahoo_fin import stock_info as si
assets = { ‘equity’:’IVV’, ’bond’:’AGG’, ’cash’:’SHY’, ’em’:’EEM’, ’dm’:’EFA’ }
prices = {k:si.get_data(v) for k,v in assets.items()}
The raw price data I downloaded includes seven fields: open, close, high, low, adjclose, volume, ticker.
I prefer to use the field adjclose as the most accurate version of total return because it counts for the reinvestment of dividends and adjusted historical prices for all corporate actions.
# transfer and plot total retun pricestotidx = pd.DataFrame.from_dict({
k:v.adjclose for k,v in prices.items()
})\
.dropna()\
.apply(lambda x: x/x.iloc[0]*1000)totidx.plot()
plt.title(‘Total Return Index with Base Value = 1000’, {‘fontsize’:16})
plt.show()
It's also worth reviewing some quick Performance Statistics. Emerging Market (em) observed the highest return but is also the most volatile. It looks that US Equity (equity) is the best-performed asset class within our review history.
# show some basic performance statsmret = totidx.asfreq(‘m’,method=’ffill’).pct_change().dropna()
pd.DataFrame.from_dict({
‘annualized monthly return’:mret.mean()*12*100,
‘annualized monthly volatility’:mret.std()*np.sqrt(12)*100,
‘annualized monthly sharpe’:mret.mean() / mret.std() * np.sqrt(12),
}).T.round(2)
The Simplest Form of Portfolio
The simplest form of a portfolio is a fixed-weighted two-asset portfolio, which you only need to choose the allocation between equity and bond and keep them constant forever.
Financial advisors usually start with asking for your risk tolerance: are you more aggressive, moderately aggressive or conservative? These will be transformed into different weights between equity and bond. I will cover more about this topic in the future under the category of Robo-Advising.
# create fixed-weighted portfolios based on risk-tolerancereb_dates = totidx.asfreq(‘q’).indexweights = dict()
weights[‘aggressive’]= pd.DataFrame( data={‘equity’:.8,’bond’:.2}, index=reb_dates, columns=totidx.columns ).fillna(0)
weights[‘moderate’]= pd.DataFrame( data={‘equity’:.6,’bond’:.4}, index=reb_dates, columns=totidx.columns ).fillna(0)
weights[‘conservative’]= pd.DataFrame( data={‘equity’:.4,’bond’:.6}, index=reb_dates, columns=totidx.columns ).fillna(0)# plot average weights
pd.concat({k:v.mean() for k,v in weights.items()},axis=1)\
.loc[[‘equity’,’bond’]]\
.plot.pie(autopct=’%1.1f%%’,subplots=True,legend=False)plt.suptitle(‘Average Weights with Different Risk Tolerances’)
plt.show()
Let’s see how they perform with a simple backtesting algorithm.
# simple backtest function
def backtest(w,prices, v0=1000,t=0,start_date=None):
p = prices[w.columns]
v = pd.DataFrame(data = v0*np.ones(len(p)),index=p.index)
for i in w.index:
# i = rebalance date ; t = trade offset, d = effective date
d = i + timedelta(t)
w_i = np.array(w.loc[:i].tail(1)).flatten()
v_i = np.array(v.loc[:d].tail(1)).flatten()
p_i = p.loc[d:]
v.update( (p_i/p_i.iloc[0]).dot(v_i * w_i) )
if start_date is not None:
v = v[start_date:]
v = v/v.iloc[0]*v0
return(v)
# run backtest
test1 = pd.DataFrame.from_dict({k:backtest(v,totidx)[0] for k,v in weights.items()})test1.plot()
plt.title(‘Backtest Performance of Fixed-Weighted Portfolios’,{‘fontsize’:16})
plt.show()# performance review
mret = test1.asfreq(‘m’,method=’ffill’).pct_change().dropna()
pd.DataFrame.from_dict({
‘annualized monthly return’:mret.mean()*12*100,
‘annualized monthly volatility’:mret.std()*np.sqrt(12)*100,
‘annualized monthly sharpe’:mret.mean() / mret.std() * np.sqrt(12),
}).T.round(2)
So we have an annualized return of 8.5% with 10% volatility for our aggressive portfolio composed of 80% equity and 20% bond. The result is pretty good for history including the 2008 Financial Crisis, the 2019 Trade War Saga and most-recently the CoronaVirus Outbreak Fear. If you calculate CAGR with the consideration of compounding, the return could be even better.
The More Active Asset Rotation Portfolio
Now we need to derive some valuation scores for our assets. The simplest of such models is price momentum. There are different ways to evaluate the attractiveness of assets, sectors or individual stocks. I will cover more topics about it in the future in categories such as smart beta or factor investing.
# create momentum ranksmtridx = totidx.asfreq(‘m’,method=’ffill’)[[‘equity’,’dm’,’em’,’bond’]]mranks=dict()
mranks[‘3m’]= mtridx.pct_change(3).dropna()\
.rank(axis=1,ascending=False)
mranks[‘6m’]= mtridx.pct_change(6).dropna()\
.rank(axis=1,ascending=False)
mranks[‘9m’]= mtridx.pct_change(9).dropna()\
.rank(axis=1,ascending=False)
mranks[‘12m’]= mtridx.pct_change(12).dropna()\
.rank(axis=1,ascending=False)
The next step is to convert factor scores into weights. You can do equal weights for simplicity, but here I provided sample codes that assign the top-ranked security to 40% of the total portfolio, the second to 30%, the third to 20% and the fourth to the remaining 10%.
# convert ranks to weights (1=0.4, 2=0.3, 3=0.2 , 4=0.1)
s = {1:0.4,2:0.3,3:0.2,4:0.1}def rankWeights(m,s):
w = pd.DataFrame(data = 0,index=m.index,columns=m.columns)
for k,v in s.items(): w[m==k]=v
w[‘cash’]=1-w.sum(axis=1).round(2)
return w# add weights to list
for k,v in mranks.items():
weights[k]=rankWeights(v,s).asfreq(‘q’,method=’ffill’).dropna()
The average rotation weights are similar across assets and momentum models, from 20% to 29%. This suggests that their performance should also be close to the fixed-weighted aggressive (80/20) and moderate (60/40) portfolios. But will they?
The answer is NO. Rotation portfolios’ performances are much worst than the moderate portfolio, with less return and higher volatilities.
Mean-Variance Optimized Portfolio
In the last backtest, we look at implementing the Mean-Variance (Markovitz) Optimization for portfolio construction.
Here’s the function to calculate the efficient frontier:
def eff_frontier(mu, sigma, clip=(0,1), dr=0.0025, symbols=list(mu.index)):
pfRet = lambda x: x.dot(mu)
pfVar = lambda x: x.dot(sigma).dot(x)
w = w0 = np.ones(len(mu))/len(mu)
mus = np.arange(start=mu.min(), stop=mu.max()+dr, step=dr, dtype=’float’)
results = pd.DataFrame(index=mus, columns=[‘return’,’risk’,’sharpe’] + symbols)
for r in mus:
w_constraints = (
{‘type’: ‘eq’, ‘fun’:lambda x:sum(x)-1.},
{‘type’: ‘eq’, ‘fun’:lambda x:pfRet(x)-r},
)
w_opt = minimize(pfVar, w, method=’SLSQP’, constraints=w_constraints, bounds = [clip]*len(w) )
w = w_opt.x
results.loc[r]=[ pfRet(w), np.sqrt(pfVar(w)), pfRet(w) / np.sqrt(pfVar(w)) ] + list(w)
return results.reset_index(drop=True)I also wrote a wrapper function to calculate return and covariance matrix, solve for the efficient frontier, find the optimized weights for the best Sharpe ratio, minimized volatility given expected return, and maximized return given expected risk.
def optimization(prices,clip=(0,1),exp_ret=0.07,exp_vol=0.15,freq=’m’,lookback=12,yearbase=12):
if lookback is not None:
mret = prices.asfreq(freq,method=’ffill’).pct_change().tail(lookback).dropna()
else:
mret = prices.asfreq(freq,method=’ffill’).pct_change().dropna()
mu = mret.mean()*yearbase
sigma = mret.cov()*yearbase
results = eff_frontier(mu,sigma,clip)
return {
‘eff_frontier’: results,
‘best_sharpe’: results.sort_values(by=’sharpe’,ascending=False).head(1),
‘min_vol’: results.sort_values(by=’risk’,ascending=True).loc[results[‘return’]>=exp_ret].head(1),
‘max_ret’: results.sort_values(by=’return’,ascending=False).loc[results[‘risk’]<=exp_vol].head(1),
}Here are the plots for efficient frontier and weights:
opt_weights = optimization(mtridx,exp_ret=0.02,exp_vol=0.2,lookback=None)results = opt_weights[‘eff_frontier’]
pf1 = opt_weights[‘best_sharpe’]
pf2 = opt_weights[‘min_vol’]
pf3 = opt_weights[‘max_ret’]plt.plot(results[‘risk’],results[‘return’],’ — ‘,label=’eff frontier’)
plt.scatter(pf1[‘risk’],pf1[‘return’],label=’best sharpe’,s=250,marker=’*’)
plt.scatter(pf2[‘risk’],pf2[‘return’],label=’min vol’,s=250,marker=’*’)
plt.scatter(pf3[‘risk’],pf3[‘return’],label=’max ret’,s=250,marker=’*’)
plt.title(‘Efficient Frontier’,{‘fontsize’:16})
plt.xlabel(‘Expected Return’)
plt.ylabel(‘Expected Risk’)
plt.legend()
plt.show()
results.set_index(‘return’)[mtridx.columns].plot.area()plt.axvline(x=pf1[‘return’].values,c=’black’,ls=’ — ‘)
plt.text(x=pf1[‘return’].values-0.001,y=0.6,s=’best sharpe’,rotation=’vertical’,fontsize=12)plt.axvline(x=pf2[‘return’].values,c=’black’,ls=’ — ‘)
plt.text(x=pf2[‘return’].values-0.001,y=0.6,s=’min vol’,rotation=’vertical’,fontsize=12)plt.axvline(x=pf3[‘return’].values,c=’black’,ls=’ — ‘)
plt.text(x=pf3[‘return’].values-0.001,y=0.6,s=’max ret’,rotation=’vertical’,fontsize=12)plt.title(‘Efficient Frontier Weights’)
plt.legend(bbox_to_anchor=[1.1,0.5],loc=’right’ )
plt.xlim(results[‘return’].min(),results[‘return’].max())
plt.ylim(0,1)plt.show()
Now we are ready to run the optimization backtests. Here’re the results of three portfolios with constraints of 5% minimal expected return, 10% maximal expected risk and 50% maximum single asset weight.
symbols=[‘equity’,’bond’,’dm’,’em’]reb_dates = totidx.asfreq(‘q’).index
w1=pd.DataFrame(data = 0,index=reb_dates,columns=symbols)
w2=pd.DataFrame(data = 0,index=reb_dates,columns=symbols)
w3=pd.DataFrame(data = 0,index=reb_dates,columns=symbols)for d in reb_dates:
p = totidx.loc[:d,symbols]
if len(p)>=250:
opt_w = optimization(p, clip=(0,0.5), exp_ret=0.05, exp_vol=0.10, lookback=36)
w1.loc[d] = opt_w[‘best_sharpe’][symbols].values.flatten().tolist()
w2.loc[d] = opt_w[‘min_vol’][symbols].values.flatten().tolist() if len(opt_w[‘min_vol’])>0 else 0
w3.loc[d] = opt_w[‘max_ret’][symbols].values.flatten().tolist() if len(opt_w[‘max_ret’])>0 else 0w1[‘cash’]=1-w1.sum(axis=1).round(2)
w2[‘cash’]=1-w2.sum(axis=1).round(2)
w3[‘cash’]=1-w3.sum(axis=1).round(2)weights[‘best_sharpe’]=w1
weights[‘min_vol’]=w2
weights[‘max_ret’]=w3
The optimized backtests with return and risk estimates derived from the ex-posted return history couldn’t prove that they are superior to the simple fixed-weighted portfolios. But I actually like optimization, because it’s quite challenging to do right. I’ll see if I can post some more sophisticated examples in the future.
Conclusion
I have demonstrated how to use python (without relying on any 3rd party API) to quickly test quantitative strategies.
Within the three tested portfolio construction techniques: fixed-weights, asset-rotation, and optimization, fixed-weights seem to be the winner for their simplicity, robustness and superior risk-return trade-offs. But, of course, there are lots of rooms for the other two to improve (which I hope I can address individually in the future).
My last words
Thanks for your patience to finish reading my first Medium blog. I look forward to sharing with you more interesting quantitative finance and data science topics in the future.
Best regards,
Richard
