What is Gaussian Process? [Intuitive Explaination]

I was first introduced to the concept of Gaussian processes (GP) while reading Kevin Murphy’s Machine Learning: A Probabilistic Perspective. However, the book heavily focuses on mathematical derivations and can be challenging to understand intuitively. In this blog post, I will explain the concept in a more intuitive way, so that it’s easier to understand for anyone without a strong mathematical background.

GP is a concept in probability theory and statistics, specifically in the area of stochastic processes. Let’s first take a look at the definition on Wikipedia:

A Gaussian process is a stochastic process (a collection of random variables indexed by time or space), such that every finite collection of those random variables has a multivariate normal distribution, i.e. every finite linear combination of them is normally distributed. The distribution of a Gaussian process is the joint distribution of all those (infinitely many) random variables, and as such, it is a distribution over functions with a continuous domain, e.g. time or space.

To understand GP, let’s first define what a process is. A process is a collection of random variables that are indexed by time or space. For example, the temperature at different times of the day can be considered a process. A Gaussian process is a process in which any finite set of random variables has a joint Gaussian distribution.

In simpler terms, a Gaussian process is a way of representing a function using a…