Keeping Bad Customers at Bay : Detecting Forged Aadhaar Cards using Computer Vision

Published in

Airtel Digital

7 min readMay 17, 2023

Introduction

Aadhaar card is widely considered to be the most reliable proof of identity and address for SIM card activations in India. Airtel, being the largest telecom provider in the country, handles ~400K sim-card activations every day. However, with the increasing digitisation of processes, the authenticity of digital images and documents is often questioned due to the widespread availability of sophisticated image editing tools. Forgery in Aadhaar cards involves modifying the content of an Aadhaar image or creating a fake Aadhaar using online software, which poses a serious challenge in maintaining the integrity of the SIM activation process.

Verifying the authenticity of every Aadhaar card manually on such a large scale is a monumental task, requiring significant resources and costs. Moreover, detecting and preventing forgery is a constantly evolving challenge, as the types of forgery and their locations can vary widely over time.

Hence, the need to develop an AI framework that could act as the first line of defence against such forgery attempts. Secondly, it should operate in real-time since the process of Document-KYC is evolving and becoming increasingly digitised.

Understanding Forgery Patterns

There are various forms of forgeries prevalent throughout the country. Some forgeries are smart, creating Aadhaar cards that appear identical to the original, while others are unsophisticated and can be easily recognised by human observations. Smart forgeries are difficult to detect, however forgeries that are easily identifiable by human observations can also be detected/learned by neural networks.

Below are few examples of forged templates-

The aforementioned examples are few instances of the prevalent forgery templates existing across country. There are several other types of forgeries that continue to occur across the country, frequently making headlines in the news. For instance, a recent news article highlights the case of aadhaar forgery that happened in the national capital- https://www.hindustantimes.com/cities/delhi-news/delhi-cyber-cafe-raided-fake-aadhaar-racket-busted-101675795291317.html

UIDAI has started issuing an official PVC Aadhaar card to prevent such malpractices, they have proper guidelines that specifies the complete list of security features that must be present on an official PVC Aadhaar card. Here is the link of official UIDAI website highlighting the security features on Aadhaar- https://uidai.gov.in/en/1024-english-uk/faqs/aadhaar-online-services/order-aadhaar-pvc-card-online/12505-what-are-the-security-features-of-aadhaar-pvc-card.html

Transforming Business Problem To Data Science

The objective was to develop an algorithm that could accurately identify forged Aadhaar cards while minimising false positives, as any erroneous classification could result in direct losses in customer acquisition. We addressed this challenge using image classification technique, with the goal of classifying each Aadhaar card as either genuine or forged.

This approach transforms the problem into a binary image classification task, which falls under the category of supervised learning. The task involves assigning labels to each image, indicating whether it belongs to a certain class or not.

The labels are described as below-

Forged: Aadhaar card image that has been modified by changing the contents or re-creating the entire Aadhaar template.

Genuine: Aadhaar card image that follows the guidelines and is in standard template format issued by UIDAI.

One of the main challenges in deep learning problems is obtaining labeled data. To address this, we manually labeled our dataset and then used a weak-supervised learning approach to generate additional labels. Rather than building a model from scratch to solve a similar problem, we leveraged a pre-trained model, VGG-16, as a feature extractor for transfer learning on our dataset. We fine-tuned the model by training some layers while leaving others frozen, given our dataset size was small.Our approach achieved both high precision and decent recall rates, ensuring robust forgery detection capabilities.

Why Transfer Learning Not - Traditional CNN?

Whenever we have any image classification problem at hand, the first deep learning model that comes to our mind is CNN- Convolution Neural Network. As we all know, these deep-learning neural network models are very data-hungry and need thousands or millions of data labels in order to make relevant predictions.

Thankfully, this problem is solved using Transfer learning.

The reuse of a pre-trained model on a new problem, the art of transferring an already trained deep learning model to a different but closely linked problem is known as transfer learning

How does transfer learning help?

It helps the model to leverage the knowledge learned from pre-trained tasks and achieve better performance with less training data and time.
It can also help in overcoming the problem of overfitting by providing regularisation to the model.

In our use case, we achieved higher precision both with lesser training time and labels using the transfer learning technique.There are lot of informative articles on transfer learning, its applications and industry-wise implications. You can the article for detailed understanding on transfer learning- https://www.analyticsvidhya.com/blog/2021/10/understanding-transfer-learning-for-deep-learning/

Which Pre-trained Model Performed Best?

VGG16 outperformed all other existing pre-trained convolution neural networks in our use-case.Despite being one of the most basic models, VGG16 has proven to be effective in achieving high accuracy results. VGG16 is a convolutional neural network trained on a subset of the ImageNet dataset, a collection of over 14 million images belonging to 22,000 categories. In the 2014 ImageNet Classification Challenge, VGG16 achieved a 92.7% classification accuracy. But more importantly, it has been trained on millions of images.

It takes a tensor with shape 224*224 and 3 RGB channels as input, the unique thing about vgg16 is instead of having a large number of hyper-parameters they focused on having convolution layers of 3x3 filter with stride 1 and always used the same padding and maxpool layer of 2x2 filter of stride 2.
Conv-1 Layer has 64 number of filters, Conv-2 has 128 filters, Conv-3 has 256 filters, Conv 4 and Conv 5 has 512 filters.
Three Fully-Connected (FC) layers follow a stack of convolutional layers: the first two have 4096 channels each, the third contains 1000 channels (one for each class). The final layer is the soft-max layer.

The above description provides a brief overview of the VGG16 architecture. For a more comprehensive understanding, we recommend referring to below detailed paper on VGG16-

Title: Very Deep Convolutional Networks For Large-Scale Image Recognition Authors: Karen Simonyan, Andrew Zisserman Link: https://arxiv.org/pdf/1409.1556.pdf

Our Transfer Learning approach will involve using layers that have been pre-trained on a source task in VGG16. We would typically download some pre-trained VGG16 model and “cut off” its top portion (the fully-connected layer), leaving us with only the convolutional and pooling layers. The reason for not including the top layer is because the VGG16’s fully-Connected layer generates 1,000 different output labels, whereas our Target Dataset has only two classes for prediction i.e., genuine or forged aadhaar card. We have to fine-tune the model to best fit our use case.

How can we make results better?

When using these pre-trained layers, we can decide to freeze specific layers and allow other layers to update their weights during training to improve target data classification. Freezing refers to not updating the weights of the layers during the back-propagation process.

In our process, we have trained the last convolution block and the fully connected dense layer. The output layer uses sigmoid function since target variable is binary i.e., genuine or forged

We have trained the last convolution block (3 layers) and the fully connected layers(3 layers). Rest all the layers are frozen, these layers act as a feature detector and pass the strong signals to the final and fully connected layers.

Impact of Framework

The model is currently in operation across almost all states of India, with a precision rate of 99.99%. It has been seamlessly integrated with the SIM activation application that is available to all Airtel retailers. Whenever a customer’s Aadhaar card image is scanned by a retailer for a new SIM activation, the model is automatically invoked through an API.

The model returns a score (prob) of an aadhaar card on the basis of which is accepted or rejected-

where p(x) is probability and threshold is decided on basis of hit and trials resulting in maximum precision.

The forgery model is integrated with the onboarding Airtel application used by retailer for sim activation process using Document-KYC.

Future Scope

The ability to identify forged Aadhaar cards can be extended to other documents used for the KYC process, thereby enhancing the robustness of activation process.

Conclusion

We have developed a framework that utilises deep learning techniques to efficiently block forged Aadhaar cards in the initial stage of the SIM activation process, while also reducing operational costs. In addition, this framework helps in maintaining the integrity of the sim activation process.

We recognise the significance of AI in shaping the future, and we are enthusiastic about incorporating it into our business operations and use cases. Central Data Science team at Airtel Digital is dedicated to providing the best possible customer experience by utilising a range of deep learning and machine learning techniques to improve our services, enhance brand trust, and build stronger relationships with our customers.

Author : Damini Babbar, Aditya Jain

Thanks to Dipyaman Banerjee, Ankit Sharma, Nitin Seth, Himanshu Sharma, Suresh G, Prashant Saksena for their guidance and support in this initiative.