Denoising Images Using Autoencoders

Image noise may be caused by various intrinsic or extrinsic conditions which are practically hard to deal with. Denoising images is a very fundamental challenge in the field of image processing and computer vision. Therefore, it plays an important part in a wide variety of fields where getting the original image is really important for robust performance.

Some examples of how images with noise look like

Example of image with noise

What is an Autoencoder?

Autoencoder is an unsupervised artificial neural network that is trained to copy its input to output. Let’s consider that we are given an image, an autoencoder will first encode the image into a lower-dimensional representation, then decodes the representation back to the image.

With appropriate dimensionality and sparsity constraints, autoencoders can learn data projections that are far more interesting than PCA or other basic techniques.

Autoencoders are only able to compress data similar to what they have been trained on. They are also lossy in nature which means that the output will be degraded compared to the original input.

Where else can Autoencoder be used?

1. Converting a black and white image into a colored image
2. Extract required features from an image
3. Dimensionality reduction
4. Watermark Removal

The list isn't exhaustive so let me know in the comments if you find more uses of autoencoders.

Architecture

There are mainly 3 parts in autoencoders

1. Encoder: In this part of the architecture the model compresses the input data to represent the compressed data in a reduced dimension.
2. Code: Also known as Bottleneck this part of the architecture represents the compressed data that is going to be fed to the decoder.
3. Decoder: This part reconstructs the encoded data as close to the input data as possible. The output from the decoder is a lossy reconstruction of the original data.

The goal of an autoencoder is to get an output that is identical to the input. The dimensionality of the input and output is similar as obviously the goal is to get the output as identical to the input we can get.

They are trained similarly to ANNs via backpropagation.

Autoencoder Architecture

Hyperparameters Of Autoencoders

There are mainly 4 parameters that we need to set before training the autoencoder

1. Code size: This represents the number of nodes in the middle layer. The smaller the code size more the compression is and if you want less compression then increase the code size.
2. Number of Layers: In the above architecture image there are only 2 layers in encoder and decoder but we can make it as deep we want it to be.
3. The number of nodes per layer: Usually what happens is the number of nodes per layer decreases with each subsequent layer of an encoder and then starts increasing again with each subsequent layer of the decoder. The decoder is symmetric to the structure of the encoder but that’s not a requirement.
4. Loss function: The popular choices here are mean squared error(MSE) or binary cross-entropy. If the input values are in the range [0,1] then binary cross-entropy is favored as compared to MSE. Otherwise, we just use MSE.

Now, you know the basics of the autoencoder, and if this interests you then you can find some helpful links at the end of the blog which are blogs from other writers who have driven deep in this topic and also have talked about the mathematical part of the autoencoder.

Let’s dive into building our encoder which will help us to denoise the images.

Implementation

For implementing this we will be using the famous MNIST dataset as input. For this, we don’t need to download the dataset. We can import it from Keras library.

MNIST is a dataset of black and white handwritten images of size 28x28.

Denoising is the process of removing noise. This can be an image, audio, or document. You can train an Autoencoder network to learn how to remove noise from pictures.

To train our autoencoder let’s first start by loading our data from Keras and scale down to [0,1].

Let’s now move to add noise to the dataset so that it can be fed to the autoencoder as the input data. We will be adding Gaussian noise to the image by a factor of 0.5.

Now, if we try to visualize the images we will get the images with white noise on it.

With this piece of code, we will be able to display the noised images.

Noisy Digits

These noisy digits will serve as our input data to our encoder. The autoencoder isn’t fed the original images at any point in time and we are expecting the autoencoder to give us output without any noise. Let’s see how good of work the autoencoder will do and if it is able to remove the noise.

In the first half of the code, we are defining the input shape to be (28, 28, 1) as the dimensions of the images are 28x28 and are grayscale which explains the input shape.

Then we have the convolution layer of kernel size (3, 3) with 32 output filters and activation function ‘Relu’. As padding is mentioned as the same the image size won’t be decreased and will remain the same.

After each convolution layer, we are using the MaxPooling layer to reduce the dimensions. The (28, 28, 32) output from the first convolution layer is decreased by a factor of 2 so it will be (14, 14, 32) and then again by a factor of 2 which gives (7, 7, 32).

This encodes the data and now if we move to the second part of the code it is the reconstruction of the original data. This is the part where the autoencoder learns to remove the noise from the image. We are using UpSampling to rebuild the images to the original dimension (28, 28).

Moving to the final part of the implementation let's fit the model.

And that's it! You have implemented your autoencoder which will help us in denoising the images. Let’s check out the output from the autoencoder to see if it really did a good job or not.

You can see the awesome job our autoencoder has done in reconstructing the image and give us the denoised images back. If I want to summarise the whole process in one image, the image below is the best for that.

Pat on your back for doing an awesome job! If you liked what you saw and read then don’t forget to smash that clap button and support me. You can reach out to me on LinkedIn to tell me your thoughts about the blog or just to connect.

If you want to deep diver into this particular subject then you can read it from the below links.

Building Autoencoders in Keras

Introduction to autoencoders.

Intro to Autoencoders | TensorFlow Core