An Empirical view of Marchenko-Pastur Theorem

Photo by Luca Bravo on Unsplash

Machine Learning is the new trend in finance, but it’s really tough to find quality content on the internet. Either is too abstract or too simple and it doesn’t help us with real-life problems. My goal is to write about uncovered topics, help the community improve their skills, and discuss quantitative finance on Medium.

If you already had some experience in the field, you’ve probably faced the challenge to separate noise and signal from a covariance or correlation matrix. Understanding the Marchenko-Pastur Theorem is a good way to start a denoising solution.

In this article, we’ll explore how to separate noise and signal.

What do you need to know?

It’s important to have some basic knowledge of Eigenvectors and Eigenvalues, Kernel Density Estimates (don’t freak out, it’s simple), and, of course, Python.

These articles are a good starting point:

The Jewel of the Matrix: A Deep Dive Into Eigenvalues & Eigenvectors

What are Eigenvalues and Eigenvectors?

Histograms and Kernels Density Estimates

Correlation Matrix, Signal, and Noises

Correlation Coefficient

Correlation measures the relationship between two variables. There are several ways to measure this relationship and we’re going to use the Pearson Correlation Coefficient.

Pearson Correlation Coefficient

If you didn’t understand the formula, you can check this nice explanation of correlation metrics.

Pearson Correlation, a Mathematical Understanding!

Measuring the relationship between a set of variables, you’ll get the correlation matrix.

Noise and Signal

In systematic investing, covariance matrix is one of the key instruments to build a portfolio. Unfortunately, we can’t just calculate it without treating some known problems. Roughly speaking, the cov matrix contains noise and signal and it’s necessary to separate it from each other to get more realistic results.

If we don’t do this, the results probably will be misleading and our conclusions will be based on information that happened randomly. Remember, we want to find patterns, randomness is a bad thing here.

Now your question probably is: How do I separate noise from signal?

A series with noise and denoised (only signal)

Marchenko-Pastur Theorem

Now that we understand what is noise, signal, eigenvalues, eigenvectors, correlation, and covariance, it’s time to go further.

Besides the fancy name, the theorem it’s not scary as it looks like. First, I’m going to show you the theorem applied in a random matrix and then to a matrix with noise and signal.

Consider a random matrix whose entries are independent identically distributed random variables with mean 0 and variance σ². The eigenvalues λ from a matrix

asymptotically converge to this: N, T → +∞ with 1 < T/N < +∞

The Marchenko-Pastur probability density function (PDF) will be:

Marchenko-Pastur Probability Density Function

Where:

Maximum and minimum expected eigenvalues

We can consider that eigenvalues that are in the range of the maximum and minimum eigenvalues are consistent with random behavior, the remaining is considered signal.

Hands-on

Let’s implement this in python and plot a chart with the output:

Marchenko-Pastur PDF Python function

Getting Eigenvalues and Eigenvectors from a Hermitian Matrix

Kernel Density Estimation

Ok. Now we have all the functions necessary to go on.

The next step is:

• Generate a random matrix
• Calculate the correlation matrix
• Get the Eigenvalues and Eigenvectors
• Calculate the Marchenko-Pastur PDF and an empirical KDE

Here we go:

Proving Marchenko-Pastur

The output is:

Marchenko-Pastur: Theorem vs Empirical Value

As we can see, the empirical result similar to the theory. So we can say that MP can be applied in the real world.

You’ve noticed that, in this example, we only have noise. What if we test the same thing with a matrix with signal? This is our final step.

Now we’re going to:

• Create a matrix random matrix
• Calculate the covariation matrix
• Add a signal to the covariation matrix
• Convert to correlation
• Get the max eigenvalue and the noise variance
• Calculate the Empirical and MP PDF

Plotting the result:

Empirical PDF for a Random Matrix with Signal

Everything on the right side of the chart is an eigenvalue associated with signal (i.e. cannot be associated with noise), if you run the code you’ll see that are exactly 100 points (nFact), which is our signal. Another interesting conclusion is that 1 minus σ² is the percentage of variance that can be attributed to signal. This is a way to measure signal to noise, but this is a topic for the next conversation.

References

López de Prado, M. (2020). Machine Learning for Asset Managers (Elements in Quantitative Finance). Cambridge: Cambridge University Press. doi:10.1017/9781108883658

Wikipedia contributors. (2020, June 14). Marchenko — Pastur distribution. In Wikipedia, The Free Encyclopedia. Retrieved 04:45, June 20, 2020, from https://en.wikipedia.org/w/index.php?title=Marchenko%E2%80%93Pastur_distribution&oldid=962460777