Understanding the Structure of RGB Images and How Pixel Values Represent Color

This article explains how RGB images are represented as matrices and how a single pixel value represents colour.

There are several image formats, such as RGB, HSV or CMYK, each with its own way of storing and representing images. In this article we focus on RGB images, which consist of pixels containing three color channels: red, green, and blue. These primary colors can be mixed in various proportions to generate a diverse array of colors. The specific levels of red, green, and blue in a pixel define its resulting color.

Let’s have an example. In the illustration above, imagine the picture on the left only has a size of 5x5 pixels. Thus, the image consist of 25 pixels in total. In reality, we would barley see a picture this small, but it is a good size for illustration purposes. As you can see the image has a grid. This grid can be used to access each pixel. On the right, we see how the python library OpenCV (cv2) stores this particular picture, namley in matrices with a shape of [5,5,3]. The last index (3) indicates the three different colours Red, Green, Blue. If we now access one particular pixel, for instance at location [4,4] we receive the pixel values [24,23,34]. The first value depicts the intensity for the colour red, the second one represents the intensity for the colour green and the last one for blue. In combination, those three colours yield a new colour which is depicted at this particular pixel location. Those values range from 0–255.

OpenCV versus PyTorch Image representation:

OpenCV:  [width, height, colour channel]
Pytorch: [colour channel, width, height]

How to open Images in Python with OpenCV and output the RGB matrices

Let’s do this in python with the cv2 library and a new image as example.

import cv2
import matplotlib.pyplot as plt

img = cv2.imread('image.jpg') #load the image
print(f'Original shape of the Image: {img.shape}') #Understand the shape of the image

Original shape of the Image: (1414, 2200, 3)

First, we open the picture of the landscape with cv2 and print the matrices. One important thing to note is that cv2 works with BGR (Blue, Green, Red) instead of RGB. However, this mainly affects the order of the third dimension of the matrices. Lets have a look. First we resize the image to (5,5), otherwise the output would be too large.

# Resize the image to 5x5 pixels 
new_size = (5, 5)
resized_image = cv2.resize(img, new_size)
print(f'new shape: {resized_image.shape}')
print(f'BGR format: \n {resized_image}') #OpenCV uses BGR instead of RGB

Output:

New shape: (5, 5, 3)
BGR format: 
 [[[150 123 107]
  [140 122 110]
  [148 114  78]
  [127 115 110]
  [ 47  39  26]]

 [[198 182 166]
  [ 73  54  44]
  [109 102 105]
  [ 30  28  24]
  [ 35  48  24]]

 [[ 63  58  72]
  [118 131 162]
  [130 130 144]
  [ 36  35   5]
  [  8  24  14]]

 [[ 54  59  44]
  [ 35  26  11]
  [ 59  59  26]
  [ 46  51  54]
  [ 13  29   8]]

 [[  0   2   1]
  [ 89 115  95]
  [ 10  17   0]
  [  0   0   0]
  [ 28  52   6]]]

The output above is how the image is stored as numbers. The first row [150, 123, 107] are the colour values for the first pixel in the top left corner of the image. Because, cv2 uses BGR, the first pixel [150, 123, 107] now represents the colours Blue: 150, Green: 123, Red:107. We can simply change the order to RGB:

#Print matrices in RGB format: 
img_rgb = cv2.cvtColor(resized_image, cv2.COLOR_BGR2RGB) 
print(f'RGB format: \n {img_rgb}')

RGB format: 
 [[[107 123 150]
  [110 122 140]
  [ 78 114 148]
  [110 115 127]
  [ 26  39  47]]

 [[166 182 198]
  [ 44  54  73]
  [105 102 109]
  [ 24  28  30]
  [ 24  48  35]]

 [[ 72  58  63]
  [162 131 118]
  [144 130 130]
  [  5  35  36]
  [ 14  24   8]]

 [[ 44  59  54]
  [ 11  26  35]
  [ 26  59  59]
  [ 54  51  46]
  [  8  29  13]]

 [[  1   2   0]
  [ 95 115  89]
  [  0  17  10]
  [  0   0   0]
  [  6  52  28]]]

We can see that only the order is affected, thus the first pixel colour values are now swapped to [107, 123, 150]. Thus, 107 is the colour intensity for red, 123 for green and 150 for blue.

Illustrating the colour of a single pixel value.

What colour exactly is represented by [107 123 150]? Well, lets check it out:

pixel_values = [107/255, 123/255, 150/255]
'''
We need to divide each value by 255 to normalise the values 
because Matplotlip only works with colour values ranging from 0 to 1
'''
# Create a square patch with the chosen color
patch = plt.Rectangle((0, 0), 1, 1, color=pixel_values)

# Create a plot with the colored patch
fig, ax = plt.subplots()
ax.add_patch(patch)
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.axis('off')

# Show the plot
plt.show()

The output is: