Automated Inference on Criminality using Face Images

Synced
Synced
Nov 24, 2017 · 8 min read

In this paper, the authors build four classifiers which are the logistic regression, KNN, SVM, CNN by supervised machine learning to discriminate between criminals and non-criminals. There are 1856 real people’s facial images controlled for face, gender, age and facial expressions. The authors find that there are some discriminating structural features can help predict criminality such as eye inner corner distance and lip curvature. Upon further study, the authors found there is a large difference between criminals and normal people in facial expressions.

I. Background

II. Data preparation

Image for post
Image for post

For these ID photos, the authors extract the region of the face and upper neck, and the background is removed. All these faces are normalized into 80 * 80 images. They also take extra measures to neutralize any other varied illumination conditions’ possible effects.

III. Implementation of Face Classifiers on Criminality and its Validation

Methods

  1. Facial landmark points, like corners of the eye and mouth
  2. Facial feature vector generated by modular PCA [5]
  3. Facial feature vector based on Local Binary Pattern (LBP) histograms [6]
  4. The concatenation of the above three feature vectors.

The criminal subset Sc is defined as positive class and the non-criminal subset Sn is defined as the negative class. The authors run 10-fold cross validation for all possible combinations of the first three feature-driven classifiers, with four types of feature vectors and one data-driven CNN without explicit feature vector. They examine the rate to classify a member of S into Sn or Sc, and average the rates of each case over ten runs in each of these 130 experiments (13 cases * 10 runs).

Results

Image for post
Image for post

We can see the accuracies of all four classifiers for the thirteen cases in Figure 2. CNN classifier achieves 89.51% accuracy. The authors also plot the ROC curves for these four classifiers in Figure 3, and give the corresponding AUC results in Table 1. This can help measure the sensitivities of the data-driven and binary face classifiers for criminality. By far, the authors can say that the predictive power of this proposed approach is established.

Image for post
Image for post
Image for post
Image for post

Validation

These experiments show that the good accuracies of the four evaluated classifiers are not due to data overfitting.

IV. Discrimination Features and Clustering of Face on Manifolds

Image for post
Image for post

The authors use Hellinger distance [8], which shows the relationship between two histograms and ranges from 0 to 1, to examine the two histograms. The histograms of the three critical features are shown in Figure 5. The mean and variance of these also are tabulated in Table 2. For angle θ, the average is 19.6% smaller for criminals than non-criminals. Similarly, For the upper lip curvature ρ, the average is 23.4% larger for criminals than for non-criminals. And the distance d for criminals is slightly shorter.

Image for post
Image for post
Image for post
Image for post

The authors also generate average faces for criminals and non-criminals. It can be seen in Figure 6. But we can find that the average faces of these two datasets are very similar.

Image for post
Image for post

To explain this phenomenon, the authors give an assumption that faces of these two datasets are assumed to populate two distinctive manifolds. They compute the cross-class average manifold distance Dx between these two subsets, and in-class average manifold distances Dc and Dn in Function 2.

Image for post
Image for post

The results show that Dc > Dx > Dn, which means the two manifolds of these two datasets are concentric. Figure 7 shows the relationship of residual variance and Isomap dimensionality. This indicates that the original ultrahigh dimensional data set in a subspace of four to six dimensions can represent itself well.

Image for post
Image for post

We can also see the data clouds of criminals and non-criminals in Figure 8. These Figures and analysis proved that there is no subjectively meaningful typical face of criminals.

Image for post
Image for post

In Figure 9, it shows four subtypes of criminal faces in Sc and three subtypes of non-criminal faces in Sn.

Image for post
Image for post

The authors also asked 50 Chinese students to separate the criminals and non-criminals in Figure 9, and results turned out to match the results the authors expected. Figure 10 shows the relationship of variation within a cluster and number of clusters for the criminal and non-criminal dataset. It clearly illustrates that before K = 4, there are four well separable clusters of criminal faces. While for non-criminals, it doesn’t form as many separable clusters in the geodesic distance. This data analysis helped the authors to draw a conclusion that criminals have greater variations in facial appearance than the general public, although they are a small minority in the total population.

Image for post
Image for post

VI. Conclusion

Before I read the whole paper, I already saw a lot of comments for this paper. One of the most famous comments is an article named “Physiognomy’s New Clothes” [9]. The scientists in that article regarded this paper as scientific racism. In my opinion, from a methodological point of view, this article has merit. After achieving relatively good results, the authors didn’t just stop but instead went on exploring the essentials of this problems. And the authors use mathematical methods to test and support the speculations. I don’t think this can be seen at an ethical level. But in my opinion, using ID photos as a dataset doesn’t make sense. Imagine the conditions that we take ID photos, there are some restrictions due to the place (the police office) we take the photos, people will be more cautious. While the more effective way to detect criminals should focus on their actions, and the demeanor and expressions when they take these actions. I think this will be more meaningful.

VII. Reference


Author: Shixin Gu | Reviewer: Haojin | Source: https://arxiv.org/pdf/1611.04135v1.pdf

SyncedReview

We produce professional, authoritative, and…

Synced

Written by

Synced

AI Technology & Industry Review — syncedreview.com | Newsletter: http://bit.ly/2IYL6Y2 | Share My Research http://bit.ly/2TrUPMI | Twitter: @Synced_Global

SyncedReview

We produce professional, authoritative, and thought-provoking content relating to artificial intelligence, machine intelligence, emerging technologies and industrial insights.

Synced

Written by

Synced

AI Technology & Industry Review — syncedreview.com | Newsletter: http://bit.ly/2IYL6Y2 | Share My Research http://bit.ly/2TrUPMI | Twitter: @Synced_Global

SyncedReview

We produce professional, authoritative, and thought-provoking content relating to artificial intelligence, machine intelligence, emerging technologies and industrial insights.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store