Portfolio Optimization using Python and CVXPY – How to select your MPF portfolio wisely?
A practical example of how you can construct well-diversified portfolios minimizing the risk using Python and CVXPY
Preface
I am new to Medium as a blogger, and also an enthusiast who always wants to share my thoughts/ideas on the topics of Quantitative Finance, Data Science, Reinforcement Learning, and Mathematics. I am working as an AI researcher in the engineering field and studying for a master’s in Data Science. One of my resolutions in 2022 was to write blogs relating to the topics above. I’m glad that I finally started in early 2023 at last haha. Hopefully, this article can bring you some thoughts on how you actualize the mathematical applications learned in school and get your hands really dirty. :D
For this topic, I did encounter the issue when I was a fresh graduate and I had to pick some funds from a “highly comprehensive” (wordy) summary recording the crazily detailed performance of each fund. At last, I followed Pareto’s Rule aka the 80–20 rule without any prior financial knowledge but with my utmost confidence from nowhere.
Introduction
The Mandatory Provident Fund (MPF) is a compulsory scheme to save money for retirement in Hong Kong. Most employees and employers are legally required to contribute monthly to the MPF, provided by approved private financial corporates.
People have to take 5% (or more) every month from their salary and their employers contribute the same percentage to the MPF. For instance, if a person earns $20,000 a month, he has to pay 5% of his salary, i.e. $1,000, and his employer also has to do the same. So, there is $2,000 cash flowing into the MPF account every month.
They have to select their own portfolios based on their risk tolerance levels and their expectations of return. According to the Mandatory Provident Fund Schemes Authority (MPFA), they suggest 80–20 (80% of the portfolio is contributed to the Equity Fund, known to be riskier but higher in return and the rest is invested into the Bond Fund) for the aggressive people, or, the more conservative contributors can consider a larger portion in a less risky fund such as Hong Kong Bond and US Bond.
Us, as retail investors, we do not want to put too much effort into our MPF and constantly change the universe of assets. In the meantime, we are also risk-averse and like to earn as much as possible. In fact, we can use past information as a reference to construct the portfolio, assuming the fund continues to behave well if it behaved well in the past.
Before introducing the methodology, I would like to ask some questions: How effective is your current portfolio? Is it possible to earn more while bearing the risk or volatility?
Prerequisite
To make my passage more friendly to readers, I think it’s a must to briefly introduce some terms, formulas, and theories to you before we start.
The expected return and standard deviation are two statistical measures that can be used to analyze a portfolio. The expected return of a portfolio is the anticipated amount of returns that a portfolio may generate, making it the mean (average) of the portfolio’s possible return distribution. The standard deviation of a portfolio, on the other hand, measures the amount that the returns deviate from its mean, making it a proxy for the portfolio’s risk.
Portfolio variance is used to measure how the aggregated returns of a set of securities making up a portfolio fluctuate over time. We normally calculate the covariance matrix to capture the standard deviation as well as the correlation of each security pair in the portfolio. We then combine it with the weight and calculate the portfolio variance (it’s mentioned below when talking about the risk minimization problem).
Convexity is the most fundamental and essential concept in convex optimization. It can be used to describe a set of values or a function.
Definition of a convex set:
Let me give a simple example of a convex set. In geometry, for a 2D space, you can select any two points in a convex set and connect them with a continuous line. On the continuous line, you arbitrarily pick a point and that point is still in the set.
Definition of a convex function:
You can draw a line, connecting any two points of a convex function, and the line is always on top of the line of the function or superposing on the line of the function.
The convex (concave) function is a function with a positive (negative) curvature that allows you to get a set of optimal point(s) and minimized (maximized) value(s) of the function.
The Sharpe ratio is a metric to measure risk-adjusted relative returns. It compares the return of an investment relative to an investment benchmark with its risk:
Markowitz Portfolio Theory (MPT)
Harry Markowitz is an American economist who was awarded a Nobel Prize for his contribution to modern portfolio theory. He introduced the MPT in his paper “Portfolio Selection” and published it in 1952 in the Journal of Finance.
It is a practical and mathematical method to select investments based on the consideration of acceptable risk and return. Using the expected return as well as the corresponding risk exposure of each combination of assets, you can visualize the following Markowitz model.
The efficient frontier, drawn by the orange line, is the set of optimal portfolios that offer the highest expected return for a defined level of risk or the lowest risk for a given level of expected return. In other words, it is the most efficient and effective way to invest with a lower risk level when you are earning roughly the same amount of expected return.
For instance, in the graph above, we can see that you can earn roughly 0.01 expectedly as a safe player with the lowest risk tolerated (around 0.475), while you can be suffering from a much larger risk (around 0.71) but the same expectation.
The Markowitz model suggests that we can (or we have to) earn more expectedly when we are able to take more risk, and why portfolio diversification plays a very important role in investment as the risk will be well diversified. So, never put all your eggs in one bucket.
Portfolio Optimization Problem
There are basically two ways to optimize your portfolio: either minimizing the risk or maximizing the expected return. I am going to include the simplest forms of both problem statements.
The objective function or loss function is the goal that you want to achieve. It’s required to be convex for a minimization problem or concave for a maximization problem. The constraints are limitations of the values or parameters that are added to the problem and are also required to be a convex function. The domain, which is the possible value of the variable in the problem, has to be a convex set.
Supposed there are K available assets. w and r are K-dimensional vectors, representing weights and expected returns of K assets respectively, and Σ is a K x K covariance matrix, storing the volatilities and correlations of the asset pairs.
Problem 1: Maximizing the expected return (Quadratic Constrained Quadratic Programming)
The objective function is an affine function of w and it is convex. We are going to maximize the expected return of the portfolio.
For the first quadratic constraint, Σ is always a positive semi-definite matrix. It, therefore, is a convex function of w. Notice that the σ² max is the maximum tolerable risk of the investor.
For the second and the third constraints, since we have to spend all our money on the MPF investment, the weight vector must be elementarily sum to 1 and every weight in the vector must be greater than or equal to 0. It is a long-only portfolio. The domain of w is a half-space which is a convex set.
Problem 2: Minimizing the portfolio variance aka the risk (Quadratic Programming)
The differences between the problems are:
- The objective function becomes minimizing the portfolio risk. It is a quadratic function of w which is a convex function.
- The first constraint of problem 2 is to set an acceptable expected return of the portfolio and the feasible w must satisfy that the dot product must be greater than the r min. It is an affine function of w and it is convex.
Implementation
Being a heavily risk-averse person, I will demonstrate how to construct the risk-minimizing problem by using python and the CVXPY library. The data is 10 years of daily prices of every fund provided by Manulife (since I have an unmanaged MPF portfolio in Manulife because of my laziness :( ) and downloaded on the Bloomberg Terminal.
Load the libraries
import numpy as np
import cvxpy as cp
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as pltRead the data
df = pd.read_csv('MANULIFE_MPF_10Y_DATA.csv')
# Equity Funds
eq_funds = ['MAMMGCV', 'MAMMGHC', 'MAMMGEE', 'MAMMGNE',
'MAMMGHE', 'MAMMGHS', 'MAMMGPE']
# Bond Funds
bd_funds = ['MAMMGPB', 'MAMMGCP', 'MAMMGHB', 'MAMMGIB']
# choose the columns I want to keep
df = df[['Date'] + eq_funds + bd_funds]
df
To construct the return vector, containing the expected return of each fund, and the covariance matrix of the funds, storing the information of volatilities:
daily_ret_matrix = df[eq_funds + bd_funds].pct_change(-1).dropna()
# expected daily return and covariance matrix
daily_r_vec = daily_ret_matrix.mean(axis=0)
daily_r_cov = daily_ret_matrix.cov()
# turn into annual data
r_annualized = daily_r_vec * 252
cov_annualized = np.array(daily_r_cov * 252)Conducting a simulation with 10000 pairs of randomized weights, we can then evaluate the performances based on the Sharpe ratio and the Markowitz model.
ret = []
std = []
num_simulation = 10000
num_funds = len(daily_r_vec)
W = np.random.normal(1, 3, (num_simulation, num_funds))
W[W<0] = 0 # long only
for i in range(num_simulation):
w = W[i]
if sum(w):
# set the weight vector elementarily sum to 1
w = w * (1 / np.sum(w))
else:
w = np.random.uniform(0, 1, (num_funds,))
w = w * (1 / np.sum(w))
ret.append(w @ r_annualized)
std.append((w @ cov_annualized @ w.T) ** 0.5)
ret = np.array(ret)
std = np.array(std)
# the current 10 years Treasury rate as the risk free rate (rf)
rf = 0.0339
sharpe = (ret - rf) / std
print('Desriptive summary of the sharpe ratios of the randomly generated portfolios')
print(pd.Series(sharpe).describe())
print('Best performance among the randomized weights\n')
print(f' Sharpe Ratio: {np.max(sharpe):.4f}')
print(f' Annualized Expected Return (%): {ret[np.argmax(sharpe)]:.4f}')
print(f' Annualized Risk (%): {std[np.argmax(sharpe)]:.4f}')
plt.figure()
plt.scatter(std, ret)
plt.ylabel('Expected Return (%)', fontsize=14)
plt.xlabel('Risk (%)', fontsize=14)
plt.show()
Construct the risk minimization problem (expecting the annualized return to be at least 0.09)
# set your own parameter: e.g. my expected min return is 0.09% per year
r_min_annualized = 0.09
# create cvxpy variable to minimize
w = cp.Variable(num_funds)
r_min = r_min_annualized / 252
# construct the objective function and constraints
obj = cp.Minimize(w.T @ cov_annualized @ w)
const = [
cp.sum(w) == 1, w >= 0,
w.T @ r_annualized- r_min >= 0
]
# solve it!
prob = cp.Problem(obj, const)
opt_v = prob.solve()
risk_opt = (opt_v * 252) ** 0.5
w_opt = w.value
print('optimal risk (%):', risk_opt)
print('optimal proportion:', np.round(w_opt, 4))
You can see that if we want 0.09 expected annualized return, according to the optimal risk, the portfolio belongs to class 5.
Backtesting result (initial capital is $100)
notional = 100
cumprod_mat = (ret_vec + 1).iloc[::-1].cumprod().iloc[::-1]
# for agggressively weighted (advice from MPFA :) fingercrossed my friend)
w_agg = np.array([0.8, 0, 0, 0, 0, 0, 0, 0.2, 0, 0, 0])
agg_cumprod = (cumprod_mat @ (w_agg * notional))
agg_cumprod = agg_cumprod.iloc[::-1].reset_index(drop=True)
# for equally weighted
w_equal = np.array([1 / len(r_annualized)] * len(r_annualized))
equal_cumprod = (cumprod_mat @ (w_equal * notional))
equal_cumprod = equal_cumprod.iloc[::-1].reset_index(drop=True)
# for optimially weighted
w_opt = w.value
opt_cumprod = (cumprod_mat @ (w_opt * notional))
opt_cumprod = opt_cumprod.iloc[::-1].reset_index(drop=True)
# date
date = df.Date[cumprod_mat.index]
date = pd.to_datetime(date, format="%d/%m/%Y")
date = date.iloc[::-1].reset_index(drop=True)
# plot the difference
plt.plot(date, opt_cumprod, label='opt')
plt.plot(date, equal_cumprod, label='eql')
plt.plot(date, agg_cumprod, label='agg')
plt.yscale('log')
plt.ylabel('Cummulative Wealth', fontsize=14)
plt.xlabel('Date', fontsize=14)
plt.legend()
Conclusion
Portfolio optimization through modern portfolio theory can be an effective way to achieve higher returns and better risk management. In this analysis, we compared three MPF portfolios based on historical returns and volatility, and found that the optimally-weighted portfolio outperformed the other two portfolios by almost twice the cumulative return over the past 10 years. Additionally, the Sharpe ratio of the optimally-weighted portfolio was 9 times larger than the Sharpe ratios of the other two portfolios, indicating better risk-adjusted returns.
However, it’s important to note that modern portfolio theory has some drawbacks, including the fact that portfolios are assessed on variance rather than downside risk. This means that while portfolio optimization can improve overall performance, it may not necessarily protect against extreme market events or downside risks. Furthermore, it’s important to remember that past performance is not necessarily a guide to future performance. Therefore, the results of this analysis should be taken as a reference for selecting a better MPF portfolio in the future rather than as a guarantee of future returns.
Remember that:
“All models are wrong, but some are useful.” George Box.
While modern portfolio theory has limitations, it remains a useful tool for investors looking to diversify their portfolios and improve risk management.
Hope you guys all enjoy the article :) Please let me know if you have any questions.
