Prospering in Image Processing

I had my first class in Image Processing last November 2020, with the hopes of learning a lot about how to understand how images work, and how they are transformed to drive out essential business value to the customer.

We discussed a lot of terms and references during the first meeting and I’m here to share with you some of the things that I have learned in class:

Image repair: example of what we can do with image processing. Image from https://sensip.engineering.asu.edu/research/image-and-video-processing/

Image Analysis Enables the following:

1. Visualization — A person is able to see referenced objects through the image at hand
2. Enhancement — A person can improve certain elements of an image by using mathematical transformations on the image
3. Extraction — A person is able to draw out useful information based on the analysis done on an image
4. Recognition — A person will be able to identify similar objects that he or she has already seen previously since the patterns or objects on the image resemble past exposure from previous images

Let’s try out sample image visualizations using some Python code:

#importing libraries
import numpy as np
from skimage.io import imshow, imread

Digital images are literally digital representations of images, which is why I imported the numpy library for this case. In reality, the digital image is represented by matrices with numerical values (which we commonly call as pixels). Translating this into example, we have this:

array = np.array([[255, 0], 
                 [0, 255]])
print(array)

Using this sample code, our output shows this array:

[[255   0]
 [  0 255]]

Which can then be shown as an image with the following code

imshow(array, cmap=’gray’);

Image representation of the generated matrix

Let’s look at the following codes with an actual image at hand:

puppers = imread('friemds.png')
imshow(puppers)

Such cute puppers

As mentioned earlier, the image is an actual digital representation, where it is composed of matrices. We can verify that by calling the puppers variable

puppers

array([[[255, 255, 255],
        [255, 255, 255],
        [255, 255, 255],
        ...,
        [255, 255, 255],
        [255, 255, 255],
        [255, 255, 255]],

       [[255, 255, 255],
        [255, 255, 255],
        [255, 255, 255],
        ...,
        [255, 255, 255],
        [255, 255, 255],
        [255, 255, 255]],

       [[255, 255, 255],
        [255, 255, 255],
        [255, 255, 255],
        ...,
        [255, 255, 255],
        [255, 255, 255],
        [255, 255, 255]],

       ...,

       [[255, 255, 255],
        [255, 255, 255],
        [255, 255, 255],
        ...,
        [255, 255, 255],
        [255, 255, 255],
        [255, 255, 255]],

       [[255, 255, 255],
        [255, 255, 255],
        [255, 255, 255],
        ...,
        [255, 255, 255],
        [255, 255, 255],
        [255, 255, 255]],

       [[255, 255, 255],
        [255, 255, 255],
        [255, 255, 255],
        ...,
        [255, 255, 255],
        [255, 255, 255],
        [255, 255, 255]]], dtype=uint8)

In order to properly digitize images, there are some techniques that can be used to fit the use case of the image being processed. Some of which are the following:

Sampling — Taking the value of an image at regular intervals

Let’s take a look at this circle in which we will try out sampling.

def circle_image(x, y):
    X, Y = np.meshgrid(x, y)
    return X**2 + Y**2
Ns = 2**np.arange(1, 5)
fig, ax = plt.subplots(1, len(Ns), figsize=(12, 4))
for i, N in enumerate(Ns):
    image = circle_image(np.linspace(-1, 1, num=N), np.linspace(-1, 1, num=N))
    ax[i].imshow(image,cmap='gray')
    ax[i].set_title(r'$N = %d$' % N)

How a circle looks like when run in code

As you can see here, the shape of the circle becomes more evident as N increases. This indicates that sampled images have denser points and better resolution as the number of pixels per side of the image increase.

Trying it out on the puppers variable, we get the following results:

factors = 2**np.arange(0, 5)
fig, ax = plt.subplots(1, len(factors), figsize=(12, 4))
for i,factor in enumerate(factors):
    image = downscale_local_mean(puppers, 
                                 factors=(factor, factor, 1)).astype(int)
    ax_num = len(factors) - i - 1
    ax[ax_num].imshow(image)
    ax[ax_num].set_title(r'$N=%d$' % image.shape[0])

You can see the improvement in image quality as N becomes larger

Quantization — Mapping out a large set of data into a smaller output set, usually the image’s intensity values.

Let’s try to look at that from the circle’s perspective first, and later on back to the cute puppers.

def circle_image(x, y):
    X, Y = np.meshgrid(x, y)
    return X**2 + Y**2
ks = 2**np.arange(1, 5)
circ_image = circle_image(np.linspace(0, 1, num=4), 
                          np.linspace(0, 1, num=4))
fig, ax = plt.subplots(1, len(ks), figsize=(12, 4))
for i, k in enumerate(ks):
    bins = np.linspace(0, circ_image.max(), k)
    image = np.digitize(circ_image, bins)
    image = np.vectorize(bins.tolist().__getitem__)(image-1)
    ax[i].imshow(image)
    ax[i].set_title(r'$k = %d$' % k)

As K becomes a larger value, more colors appear on the quantized figure

K here represents the number of bits used to represent the intensity value of an image’s pixel. By increasing the number of bits, more detail is shown that will properly represent a figure’s color space. The code below is shown to represent the quantization of the puppers.

ks = 2**np.arange(1, 5)
fig, ax = plt.subplots(1, len(ks), figsize=(12, 4))
for i, k in enumerate(ks):
    bins = np.linspace(0, puppers.max(), k)
    image = np.digitize(puppers, bins)
    image = np.vectorize(bins.tolist().__getitem__)(image-1).astype(int)
    ax[i].imshow(image)
    ax[i].set_title(r'$k = %d$' % k)

Image detail becomes clearer as K increases

Image scale conversion — Can be done by thresholding an image based on its intensity values

A basic example of this conversion practice is when converting an image into grayscale (black and white). We can see the concept in action below

from skimage.color import rgb2gray
pup_gray = rgb2gray(puppers)
imshow(pup_gray);

Still very good boys

Color Spaces — colored images are represented in RGB Space

The most basic example of color spaces are the three primary colors: Red, Green, and Blue (RGB). Images are also commonly represented in the RGB space. We can also segment the puppies accordingly.

fig, ax = plt.subplots(1, 3, figsize=(12,4))
ax[0].imshow(puppers[:,:,0], cmap='Reds')
ax[0].set_title('Red')
ax[1].imshow(puppers[:,:,1], cmap='Greens')
ax[1].set_title('Green')
ax[2].imshow(puppers[:,:,2], cmap='Blues')
ax[2].set_title('Blue');

And that’s it! You now have an idea of what image processing can do with our images. Visit my other stories to know what other applications are in store for this topic.