Using Deep Learning to reconstruct fingerprints — With Keras
Spoiler Alert: Autoencoders are awesome
Fingerprints are an important facet of biometric security techniques — they are unique, hard to steal and every person on Earth is born with one. (Almost everyone, see: Adermatoglyphia)
They form the basis of many criminal investigations, and are also used as proof of identity around the world. Unfortunately, what happens often is that fingerprints tend to get corrupted — partially erased, obscured by ink, missing, et al.
To this end, I wanted to test the plausibility of imputing missing/deformed fingerprint data from existing data. And thus began my foray into fingerprint reconstruction.
Problem Statement
Given an incomplete, or noisy sample of a fingerprint image, leverage the power of autoencoders to automatically reconstruct the complete fingerprint details.
My first step was to look for existing work, of course. I found a few papers which used statistical methods to reconstruct minutiae points and the ridges. However, deep learning approaches to this problem seemed sparse.
But I did discover two tutorials which dealt with exactly this problem — using convolutional autoencoders: one from Datacamp and another from KDNuggets. These helped me formulate the solution architecture itself.
Armed with this knowledge, I moved onto finding a dataset.
Dataset
A little poking about on the Internet lead me to the Sokoto Coventry Fingerprint Dataset (SOCOFing) on Kaggle. By the description on Kaggle:
This dataset consists of of 6,000 fingerprint images from 600 African subjects and contains unique attributes such as labels for gender, hand and finger name as well as synthetically altered versions with three different levels of alteration for obliteration, central rotation, and z-cut. For a complete formal description and usage policy please refer to the following paper: https://arxiv.org/abs/1807.10609
As part of preprocessing, we rescaled the images to be 100x100 and eliminated blank space around the fingerprint itself. We also experimented with using different filters on the image to improve the deep learning model’s understanding of the structure, but we are yet to find a suitable filter.
Convolutional Autoencoder
That’s a rather intimidating word, huh? Fret not, it’s easier than it sounds. Let's go through this one word at a time.
Convolution
According to Wikipedia:
In mathematics (in particular, functional analysis) convolution is a mathematical operation on two functions (f and g) that produces a third function expressing how the shape of one is modified by the other.
Let’s see if we can understand it in simpler terms.
In 2D convolution, the process is simple. We have a matrix of data (read: pixels of an image), and we wish to extract the features in it. So we choose a smaller matrix called the kernel/filter, and slide it over the larger matrix, applying a multiplication operation at each step. The way the output of multiplication is combined into one is called “pooling”, and can be done in a variety of ways, depending on our goal.
Different aspects of this algorithm can be customised, like the kernel, the pooling, the stride, the padding, et al. For a deeper dive into this topic, see this wonderful article by Towards Data Science.
Autoencoders
An autoencoder is a neural network architecture used to efficiently learn data representation in an unsupervised manner. This architecture has many uses, such as dimensionality reduction and data denoising.
Let’s write a simple Convolutional Autoencoder in Keras:
And that’s it! This model takes in a 100x100x1 array (100px *100px, 1 color channel), and outputs an array of the same shape.
The hyperparameters for this model have not been tuned, but are the result of arbitrary choices and trial-and-error. As you can see, the model is simply a series of Conv2D and MaxPooling layers, followed by Conv2D and UpSampling2D layers. Upsampling is simply the inverse of Pooling, since it increases the matrix dimensions by performing matrix multiplications.
Results
Upon training this model on the SOCOfing dataset with the Adam optimiser and Binary Cross-entropy as the loss function of choice, we saw impressive results:
Apologies for the terrible picture quality — there is only so much that a 100x100 image can do. The autoencoder has clearly removed the distortion present in the original image with a likely approximation of what should be there.
Drawbacks
Clearly, the entire image suffers a loss of quality when put through the autoencoder. An interesting alternative would be to create another model which can identify which part of the image has been distorted, and selectively apply reconstruction. Such an approach will reduce the loss of quality dramatically. However, due to the lack of a readily available dataset, we haven’t done this yet.
Conclusion
Clearly, this is a very simple approach to a very complex problem. I intend to continue working on this particular problem, while trying out different approaches. If I happen to make substantial progress, rest assured that I will update this page. Cheerio! 👋
