Gaining an Intuitive Understanding of 1-D and 2-D Gaussian Distributions Using Image Examples

Welcome to this blog, where the aim is to provide an intuitive understanding of 1-D and 2-D Gaussian distributions, focusing on visual examples and minimal mathematics.

Let’s Start !!

Colored images can be generally represented by the RGB color model which stands for Red, Green, and Blue. In this model, the colors are formed by combining different intensities of these three primary colors. Each pixel in a colored image has three channels or components corresponding to the intensities of Red, Green, and Blue.

The intensity values for each color channel usually range from 0 to 255, where 0 means no intensity and 255 means full intensity. For example, pure red would be represented as (255, 0, 0), pure green as (0, 255, 0), and pure blue as (0, 0, 255). By combining these primary colors in various proportions, we can represent a wide range of colors.

Since we have to talk about Gaussian, let’s not consider the range of 0–255. Then, what do we do? We can normalize this range such that instead of each color channel’s intensity value ranges from 0 to 1 instead of 0 to 255. This representation is known as normalized RGB representation.

So just to be clear, a red intensity value of 128 in the integer-based representation would be represented as 128 / 255 ≈ 0.502 in the decimal-based representation.

For example, there are 3 images below,

Figure 1: Images with averaged values for RGB over normalized scale

Let’s take the first image. I can see that there are so many different shades of blue. Then how come to a number B = 0.9 represented the normalized blue intensity for the first image? Well, consider it as an overall average representing the blue color in the first image. The same goes for colors red and green also. Notice that the normalized intensity for red (R) is very low (0.05). Yeah, it doesn’t look that reddish to me (like image 2, and that has the highest average red intensity. Why So?)

1-D Gaussian Distribution

Okay, let’s talk about how can we get more information about the blue color in the first image. I know it’s more bluish, but mathematically what does that mean? That can be described with the help of Gaussian Distribution. So, let’s look at the Gaussian distribution for the blue color in the first image (Figure 2).

Figure 2: Gaussian distribution over normalized blue color pixel intensities

Figure 2 explains a lot about the different shades of blue in the image. First, now we know that it’s not one value, and B = 0.8 in the first image in Figure 1, was just an average value. From the curve, intuitively we can think that there is a whole range of blue with different pixel intensities.

The curve above is called the Gaussian curve (light blue boundary), also known as a normal distribution or bell curve.

The Gaussian curve is a continuous probability distribution that describes the likelihood of different blue intensity values appearing in the image. What does it even mean? Let’s think logically.

Let us say I am saying that the mean of blue color is 0.8 in an image. Since, it is an image with natural colors, one way to think about this is that there will be a lot of shades of blue which will have their normalized blue pixel intensities around or near the value of 0.8 ( law of averages ), which means that probability of the occurrence of 0.8 or numbers near to it will be higher and that is represented by this Gaussian curve.

So, a higher value of the Gaussian curve around the mean value of 0.8 means that the probability of the likelihood of the numbers around/at 0.8 is higher than others.

Mathematically, say for one-dimensional Gaussian distribution (Gaussian distributed just over blue color is 1 dimensional) the random variable is represented by variable ‘X’, the mean is represented by ‘µ’ and the spread of the curve is represented by another term called as variance (represented by σ²). Thus Gaussian distribution can be expressed as:

• N is just a notation representing Normal distribution.
• X is a random variable ( in the above example it was the intensity of the blue color. It was random since it was varying).
• µ is mean. (0.8 in Figure 2,3).
• σ² is the variance. In simple terms, the variance tells about how much the blue intensities in the image vary or deviate from the average blue intensity (0.8).

Figure 3: Gaussian curve in terms of random variables and probability distribution

Putting it using Figure 3, ‘X’ is a random variable (shown in red). X1, X2 and X3 are specific values of random variables. P(X1), P(X2) and P(X3) are the specific values for random variables X1, X2, X3 from the continuous probability distribution (Gaussian curve)

Figure 4: Comparing the spread of normalized blue color pixel intensities in 2 images.

In Figure 4, Image a, has a higher spread of blue (The Gaussian distribution is wider). A higher spread means there’s a greater variation in blue shades across the image, ranging from lighter to darker shades of blue. Also, we can say that the blue pixel intensities are more diverse and deviate more from the mean value of 0.8 (that’s indeed the case)

In Figure 4, Image b, has a lower spread of blue (The Gaussian distribution is narrower in comparison to Image a). A narrower spread means there’s not much variation in blue shades across the image. Also, we can say that the blue pixel intensities are more uniform, their range is small and they deviate less from the mean value of 0.3

2-D Gaussian Distribution

Now, since we have cleared the basics of 1-d Gaussian distribution, let’s take it a step further and move into 2-d Gaussian distribution.

How can I represent the Gaussian distribution of say two colors together (Blue and Red, Green and Blue, or Red and Green)? What does it even mean to represent the Gaussian of two colors on the same plot? What about their interactions with each other (what happens when you mix two colors together? They form a new color, right?)

Let’s hash the 2-d Gaussian distribution with the help of two colors, say blue and green.

• Thinking logically, 1-d has a mean for the blue color, and 2-d would have two means, 1 for blue and 1 for green.
• 1-d had a variance for the blue color, and 2-d would had variances for both blue and green.

But, what happens to the distributions in 2-d (as drawn for blue only in 1-d?).

• Pretty much similar to 1-d, only thing is we will have two axes for 2 random variables now (blue and green) whereas there was only 1 axis for 1 random variable (blue) in 1-d plot .
• The probability distribution would be the combination of conditional probabilities from both random variables in 2-d (in 1-d it was just the probability distribution over 1 random variable, blue color).

How does it look?

Figure 5: 2-d Gaussian distribution over two random variables (color blue represented by X1, color green represented by X2)

Figure 5 is like an inverted cone. Since, two random variables are coming together to give us a 2-d plot, how to quantify the interaction?

Mathematically, 2-d Gaussian can be represented by the following:

• For 2-d Gaussian, we have a vector of the mean (because there are two variable variables now, instead of 1).
• For 2-d Gaussian, we have a matrix with variances. This matrix is also known as the covariance matrix.

Why covariance is a matrix, in the first place, and not a vector like the mean?

The covariance matrix is a matrix, rather than a vector because it needs to capture more than just the individual variances for each variable. It also needs to describe the correlation between the variables, which is given by the covariance terms (inverted diagonal).

The covariance terms, sigma(X1,X2) and sigma(X2,X1), give us information about the joint variability of the two variables. When the covariance is positive, it indicates that the variables tend to increase or decrease together. When the covariance is negative, it indicates that one variable tends to increase when the other decreases, and vice versa. When the covariance is zero, it suggests that there is no linear relationship between the variables.

Let’s analyze the concept of covariance terms through figures

Case 1: Mean is different, variances are the same, and co-variance is 0.

Figure 6: Top view of Case 1

From Figure 6, it can be seen that the top-view of the joint probability density function (PDF) is circular. This is because, in this case, the two variables are uncorrelated, and their variances are equal.

Uncorrelated variables mean that there is no linear relationship between them, so their covariance is 0. Since the variances of X1 and X2 are the same, the “spread” of the distribution is equal along both axes. This results in a circular shape when looking at the top view of the joint PDF.

When the variances are equal and the covariance is 0, the contour lines of the joint PDF represent circles centered around the means of the two variables.

Case 2: Mean is different, variances are different, and co-variance is 0.

Figure 7: Top view of Case 2

From Figure 7, it can be seen that the top-view of the joint probability density function (PDF) is elliptical. This is because, in this case, the two variables are uncorrelated, and their variances are different.

Uncorrelated variables mean that there is no linear relationship between them, so their covariance is 0. Since the variances of X1 and X2 are different, the “spread” of the distribution is different along both axes.

When the covariance is 0, the contour lines of the joint PDF do not show any tilt or rotation, as there is no linear relationship between the variables. However, the contour lines will be elongated along the axes with higher variance, resulting in an elliptical shape. The major and minor axes of the ellipse corresponding to the directions of the higher and lower variances, respectively.

Case 3: Mean is different, variances are the same, and co-variance is not zero.

Figure 8: Top view of Case 3

From Figure 8, it can be seen that the top-view of the joint probability density function (PDF) is a tilted ellipse.

• There is a tilt because the non-zero covariance (0.8) between X1 and X2 indicates a linear relationship between the two variables. The tilt represents the direction of the linear relationship, where positive covariance means X1 and X2 tend to increase or decrease together.
• The contour lines are elliptical because the variances of X1 and X2 are the same (both are 1), leading to an equal “spread” along both axes (case 1). However, the non-zero covariance introduces the tilt, resulting in an elliptical shape with the major and minor axes corresponding to the directions of the strongest and weakest linear relationships between the variables, respectively.

The point to note is that a positive covariance indicates that X1 and X2 tend to increase or decrease together, while a negative covariance implies that one variable tends to increase as the other decreases.

That’s all for the basics of 1-d, 2-d Gaussian Distribution.
If you like, please clap and share