Computer vision with human faces (2/3)
Geometry information from faces
In the previous post, we talked about the wide range of applications that extract information from faces for different purposes. In this post, we briefly explore the type of geometric information used for different types of applications.
Geometry-wise, there are three critical characteristics of the human face: structure, vertical symmetry, and components variability. In regards to the structure, the face has a well defined composition which means that the positions of its different components have small variance among various subjects, e.g., the mouth is close to the bottom, the eyes are close to the top, etc. Some applications take advantage of this property for synthesizing data to train neural networks [1], copying a makeup style from one face to another [2], or merely replacing faces in pictures for entertainment purposes [3].
The vertical symmetry of the human face is another salient geometric characteristic that several applications exploit. In the task of face recognition, this property has been used to reduce the effects of illumination in face recognition systems [4]. The main drawback of this property is its sensitivity to pose variations and facial expressions, an issue that dramatically reduces the performance in face recognition approaches. Several alternatives have tried to minimize these effects by using neural networks [5] and probabilistic models [6].
Finally, the variability of the face components from subject to subject (shape, relative position, relative size, etcetera) is high enough that it is possible to have approaches for face recognition [7], a task that leads to systems for authentication or re-identification. Also, the components of a face in a single subject are not static (their shapes can vary) which is evident in facial expressions. While this can be a problem for specific tasks, some applications leverage this property to recognize moods or emotions[8].
Other than facial expressions, subjects can modify the geometry of their faces by using makeup, wearing accessories like glasses and earrings, or even by changing their hairstyle. This situation evidently increases the variability of a single face, which supposes a challenge for systems that rely on the geometric information of faces. Some alternatives have addressed this problem by performing the correlation between the hard face and the modified one [9] Other approaches have focused their efforts in the detection of such accessories[10].
In this post, we explored the types of geometry information from faces and how some applications use them to solve specific tasks. In the next post, we will make a similar review of another important type of information from faces: color.
References
[1] Hu, G., Peng, X., Yang, Y., Hospedales, T. M., & Verbeek, J. (2018). Frankenstein: Learning deep face representations using small data. IEEE Transactions on Image Processing, 27(1), 293–303.
[2] Guo, D., & Sim, T. (2009, June). Digital face makeup by example. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on (pp. 73–79). IEEE.
[3] Bitouk, D., Kumar, N., Dhillon, S., Belhumeur, P., & Nayar, S. K. (2008, August). Face swapping: automatically replacing faces in photographs. In ACM Transactions on Graphics (TOG) (Vol. 27, №3, p. 39). ACM.
[4] Song, Y. J., Kim, Y. G., Chang, U. D., & Kwon, H. B. (2006). Face recognition robust to left/right shadows; facial symmetry. Pattern Recognition, 39(8), 1542–1545.
[5] Huang, F. J., Zhou, Z., Zhang, H. J., & Chen, T. (2000). Pose invariant face recognition. In Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580) (pp. 245–250). IEEE.
[6] Kanade, T., & Yamada, A. (2003, July). Multi-subregion based probabilistic approach toward pose-invariant face recognition. In Computational Intelligence in Robotics and Automation, 2003. Proceedings. 2003 IEEE International Symposium on (Vol. 2, pp. 954–959). IEEE.
[7] Ahonen, T., Hadid, A., & Pietikäinen, M. (2004, May). Face recognition with local binary patterns. In European conference on computer vision (pp. 469–481). Springer, Berlin, Heidelberg.
[8] Adolphs, R. (2002). Recognizing emotion from facial expressions: psychological and neurological mechanisms. Behavioral and cognitive neuroscience reviews, 1(1), 21–62.
[9] Guo, G., Wen, L., & Yan, S. (2014). Face authentication with makeup changes. IEEE Transactions on Circuits and Systems for Video Technology, 24(5), 814–825.
[10] Jing, Z., & Mariani, R. (2000). Glasses detection and extraction by deformable contour. In Pattern Recognition, 2000. Proceedings. 15th International Conference on (Vol. 2, pp. 933–936). IEEE.
