Filters as an image processing tool — part 1

Published in

Analytics Vidhya

9 min readJun 29, 2020

Introduction

Image processing is a fundamental field in the aspect of computer visioning and its widely concerned with giving machines the ability to understand images. There are images we do really want machines to understand but they can’t understand until basic features needed for comprehension is extracted and this can’t happen without going through certain image processing techniques.

All that image processing helps a machine to achieve is the ability to comprehend pixels and make better decision. This is done by enhancing the image for key features to be extracted from the image.

Before I define image filtering, you need to understand what signal processing is. Signal processing is an engineering sub-field that focuses on analyzing, modifying, and synthesizing signals such as sound, images, and scientific measurements.

Image filtering is a key signal processing technique that includes smoothing, sharpening, removing noise, and edge detection in an image. When we simply hear filters, what comes to mind is the removal of unwanted signal (component) from the characteristic a system exhibits.

In simple terms, a filter is a device or process that removes some unwanted components or features from a signal.

Signals could be in any form, it could be continuous or discrete. What do I mean by continuous signals and discrete signals.

A continuous signal is a signal of continuous amplitude with respect to time. A continuous signal is called analog signal. The act of processing signals of this form is called, analog signal processing (ASP).

A discrete time signal is a type of signal with valid real value occurring at distinct, separate “points in time”. A discrete signal is called a digital signal. The act of processing signals of this form is called Digital signal processing (DSP).

A filter is a device or process that removes some unwanted components or features from a signal.

An image is a typical example of a digital representation of signals. Let’s take a deep dive into digital signal processing (DSP).

Digital signal processing

Tools used for DSP includes convolution, Laplacian, z-transforms, Fourier transforms, wavelets and others. All this is utilized within the domain of time and frequency. The time and frequency domain are basically means of monitoring signals or defining the characteristics of a signal. Images are basically digitized signals, but the camera/device takes in the signal as analog, but recorded as digital via DSP. This is possible because while trying to capture the image, the pixels were sampled discretely. There is a resulting perfect replica in the image because of the sampling rate (sampling frequency) of the signal called called Nyquist.

Nyquist theorem

The Nyquist theorem is a theorem that focuses on signal regeneration via signal re-sampling frequency. The theorem basically states that, in order to recover the information in a signal, such signal has to be sampled at a frequency which is twice the frequency of operation for such a signal. For instance, if the operating frequency for signal A is 30kHz; to recover such signal, we have to sample at nothing less than 60kHz. This is the Nyquist theorem. This basically helps us to solve the problem of imperfect communication in the field of telecoms. For instance, I am able to communicate with you at location A from location B with this simple theorem in action.

What happens when we sample such message signal at a frequency less than nyquist, then we will be having a situation called aliasing. Aliasing is a distortion effect in a reconstructed signal which produces an entirely different signal different from the signal to be sampled.

Aliasing is a distortion effect in a reconstructed signal which produces an entirely different signal different from the signal to be sampled.

The illustration makes us understand the effect of this, The figure below shows a function (in gray/black) being sampled and reconstructed (in gold) at steadily increasing sample-densities/ sampling frequency, while the figure on the right shows the frequency spectrum of the gray/black function, which does not change. Now to better understand this, at the first simulation with frequency less than the nyquist frequency, we had a waveform entirely different from the waveform we want to reconstruct, but as we increase this frequency until it reaches the nyquist frequency value, we were able to replicate the waveform (this is nyquist).

Discretization of signals

Signal discretization is a means of signal/ waveform regeneration by using the nyquist sampling theorem, we are able to select discrete points from the regenerated signal such that when we link them all up, we will be able to generate the original signal. From the above illustration, the black dots in the waveform being generated are the discretized points for our signal regeneration.

Signal quantization

Quantization is the process of converting a continuous range of values into a finite range of discrete values.

Filters in image processing

Since signal processing is a big and broad field. I will be restricting this to image signal processing. Here we will be looking at various filters utilized for image processing, and how they are carried out.

Take note that, image processing filters are mainly used to suppress either the high frequencies in the image, i.e. smoothing the image, or the low frequencies, i.e. enhancing or detecting edges in the image. An image can be filtered either in the frequency or in the time domain depending on the domain of operation. Some examples of filters we would be looking at includes mean filter, median filter, Gaussian filter. I will also illustrate how you can programmatically do this using opencv in part 2 of this series.

First and foremost, I want to begin with filter illustrations being carried out in the frequency domain. The Frequency domain is a domain of signal representation/ analysis using frequency. This is quite different from the usual which is the time domain (signals are being represented with respect to time).

We will be looking into how these techniques could be used in enhancing image processing. Reason being that, image processing seems a lot more easier in the frequency domain than in the time domain.

Basic filters used for analog and digital signal processing in the frequency domain includes low pass filter (LPF), high pass filter (HPF), band pass filter (BPF) and band stop filter (BSF). There are 3 areas of concentration when applying signals to a frequency based signal. They include, pass band, transition band and stop band.

The pass band describes the only allowed frequency of operation for such system. The transition band is where we have the cut-off frequency and the stop band is the range of frequencies that needs to be rejected or filtered out.

Low pass filters

A low pass filter is a filter designed to suppress high frequency signal and give chance to only low frequency signals. This is simply illustrated in the figure above (a). For instance in image, processing, high variation in frequency is basically detected at the edges which shows high occurrence of noise in the image to be processed. A low pass filter can be applied to filter out the noises in the image from the true image signal. An example of low pass filter applied as an image processing tool includes: mean filter, median filter, Gaussian filter and others. All this can be simply illustrated using the scipy library, opencv and others in python.

High pass filter

An high pass filter basically suppresses low frequency signals and give chance to only high frequency signals. This is simply illustrated in the figure above (b). This is typically used as an image processing technique in filtering out low detailed information contained in an image from the high detailed information present in the image. This is core theory behind convolution, when you convolve a typical kernel of a specific dimension on a set of input image. This tend to remove, low detailed info’s and helps the algorithm to pay more attention to high detailed information that could enhance the performance of the algorithm to learn. It helps the image attain sharpness.

This is the fundamental reason why, when we reduce kernel sizes to making the network deeper, the network tends to pay more attention to image details than it’s previous kernel dimension.

Band pass filter

This is a combination of both low and high pass but in a unique way. This type of filter is an advance form filter applied as an image processing technique with 2 cut off frequencies. The first cut-off frequency represents the lower band frequency (that is frequencies less than this value are attenuated), while the second cut-off frequency represents the upper band frequency (that is, frequencies greater than this value are attenuated). This therefore, leaves us to a range of allowable frequencies (pass band). This is as illustrated in figure ( c) above.

Band stop filter

This is also a combination of both low and high pass but the reverse of band pass filters. It also has, 2 cut off frequencies, where the first represents the lower band and the second represents the upper band. Frequencies within the upper and lower band get attenuated, while frequencies outside the range are allowable frequencies (pass band). This is as illustrated in figure ( D) above.

Filters in Action

Now let’s check out some key filters notably used in image processing as filters. We will still take a deep study into what they actually do when applied to images. Let’s get to understand how they work first.

Mean filter

The mean filter is basically a type of sliding filter that replaces the center value in the window it’s being overlayed, with the average (mean) of all the pixel values in the window. for example, in the figure below, we do have a filter of dimension 3 by 3, moving over the window of a zero padded matrix of dimension 5 by 5. F[x, y] represented an unfiltered image, while G[x, y] is the resulting operation of mean filtered image.

Take note that in the first filter enclosed by the red box, for the first entry we have:

Entry 1: the mean of (0,0,0,0,0,0,0,0) that gives us 0.
Entry 2: the mean of (0,0,0,0,0,0,0,0,90), this gives us 10.
Entry 3: the mean of (0,0,0,0,0,0,0,90,90), this gives us 20.
Entry 4: the mean of (0,0,0,0,0,0,90,90,90), this gives us 30.

This goes on and on until the experimental matrix has been mean filtered.

The mean filter is a simple sliding-window spatial filter that replaces the center value in the window with the average (mean) of all the pixel values in the window. The window, or kernel, is usually square but can be any shape.

Median filter

This is similar in operation to the mean filter, what makes it different is the fact that the filtering operator doesn’t find the mean but the median. Here is an illustrative look at this technique. The image below shows an extracted 3 by 3 portion extracted from the main image. The extracted portion have pixel values of (101, 69,0, 56, 255, 87, 123, 96 and 157), finding the median of these values gives the filtered image a value of 96.

Take note that median filters deals with the apportioned set of pixels enclosed by the filter by finding the median value of all pixel entry that will be mapped to the filtered image.

Gaussian filter

Gaussian filter is a filter whose impulse response is a Gaussian function (or an approximation to it, since a true Gaussian response is physically unrealizable). There are 2 key params to implementing a Gaussian filter and they both include the mean and standard deviation for the distribution being considered.

Conclusion

Now we have been able to see some typical examples of filters in action. The next article will be based on how we can implement them as an image processing technique. If you do have any question for me, kindly leave them in the comment section below. For more studies of all have outlined above check out the reference links.

Do tag and share with a friend who needs to understand this. Thanks.