Combining Structure Similarity Index with Histogram Matching to Image-based Fraud Detection
Fraud detection is strategies to prevent uncovered activities in an organization, such as fraudulent credit card transactions, identify theft, insurance scams, and so on. A segment for fraud detection is identify altered or processed images using computer vision.
In this post, I will show you a basic way to image-based fraud detection combining two tools:
- Structural Similarity Index (SSIM);
- Histogram Matching.
SSIM
SSIM is a metric that extracts luminance, contrast, and structure features from images. Thus, comparing a reference image with a processed image, it is possible to measure the difference between them. It takes into account that the two images need to be from the same image capture. The result often is a value between 0 to 1, where 0 indicates no structural similarity and 1 indicates perfectly structural similarity. For more details about SSIM I suggest you read this post.
Given two images, a reference and an altered or processed one. The images are the same. However, in the processed image, I added a simple blur filter to show what happened with SSIM in this case. I will apply the SSIM in two situations: 1) comparing reference-reference images and 2) comparing reference-processed images.
The SSIM result for Fig. 1a is 1.0, while in Fig. 1b is 0.79. That’s nice!!! It is really what we want to see! The first result makes sense because we are comparing the same image, and the second result shows that the blur filter was enough to decrease the structural similarity. When the automatic fraud detection processes the second situation, it might be a problem whether the “threshold” is very high (like 0.9) to approve if the image is a fraud or not. It could generate false negatives. To figure it out, we are going to the next step of this post: histogram matching.
Histogram matching
An image histogram is an intensity distribution of an image, it can be for a grayscale or color image. Histogram matching is a “transformation” of an image using the histogram from another. In other words, histogram matching is a process that transfers the distribution of pixel intensities from image A to image B. More details are shown in this post.
Applying histogram matching for our problem allows us to obtain interesting results, as you can see in the Fig 2.
It is notable that applying histogram matching on processed image using the reference image histogram the output is an image very similar to the reference image.
Finally, comparing Fig 2 a, b and c with SSIM we have the following result:
As we can see, applying histogram matching on the processed image using the reference image histogram, the SSIM improved such that the output is almost perfectly structural similar to reference image. It is nice! This means that histogram matching was enough to figure out the blur filter added in the reference image and we reached our goal!
The combination SSIM + histogram matching might be used in different types of image contrasts. In this article, I showed just a situation when the processed image has a blur filter. Of course that in real life you need to use more tools than the ones presented here, however, it is a good start.
Conclusion
In this article, we covered a useful combination between SSIM with histogram matching to image-based fraud detection and a basic application in a processed image with blur filter.
