The Book of the Face
“Your face, my thane, is as a book where men May read strange matters.” Bill Shakespeare
It would have come as no surprise when Lady Macbeth told her husband this back in medieval Scotland, because as human beings we are well aware of how our faces give away our thoughts and feelings. In fact we are so well adapted to interpreting these that the inability to do so in individuals with autism can be crippling in going about their day to day life’s. Given this the question becomes, is it possible to train a computer to come anywhere close to recognising emotions like a human can.
So what are facial expressions anyway
A facial expression is one or more motions or positions of the muscles beneath the skin of the face. It is the combination of these movements which convey the emotional state of an individual to the observer. Facial expressions are a form of nonverbal communication. Nevertheless, they are a primary means of conveying social information between humans, but they also occur in most other mammals and some other animal species.
Beyond the Looking Glass
There are two main approaches to facial expression measurement in the field of psychology:
- Message Judgement
- Sign Judgement.
Message judgement aims to infer what underlies a displayed facial expression, beyond the looking glass if you may, to understand the personality or mental state that exists.
Sign judgement describes the surface of the displayed behaviour, such as facial movement or facial component shape, with the Facial Action Coding System (FACS). However the really interesting work exists where the two collide, extensive research and study has gone into undertaking a message judgement interpretation using the description of FACS as a guide. Or in plainer English using distinctive muscle movements and positions as a guide to inferring emotional states.
Working with individual muscle movements
The FACS coding of facial movements mentioned above boils down to a series of action units (AU’s) which are the fundamental actions of muscles or groups of muscles. For example, expressions typically associated with happiness contain AU6 and AU12, or as they are better known (AU6 cheek raising) & (AU12 lip corner raising). But this isn’t the full story it isn’t enough just to notice a raising of the cheeks, the dynamics speed and acceleration play an important role in the interpretation of human facial behaviour.
It is the dynamics and subtleties of these action units that mean something to humans. For example, psychologists have found a difference in duration and smoothness between spontaneous and deliberate expressions, e.g. between polite and amused smiles.
In fact facial expression dynamics are also essential for categorisation of complex mental states such as various types of pain and mood.
The forefront of emotion measurement
Given the importance of dynamic action units CrowdEmotion has sought to build a system that considers how your face changes over time. This is an improvement on approaches to date, where facial emotion recognition systems use only static appearance descriptors, which means the appearance changes and their associated temporal information are completely ignored.
We have applied a novel approach to explicit analysis of the temporal dynamics of facial actions using the dynamic appearance descriptor, it now seems natural to extend our work to start understanding the dynamic characteristics such as the intensity of Action Units and their frequency of occurrence. With this approach we can make progress in understanding the fleeting and lingering expressions that betray peoples emotions.
Then perhaps lady Macbeth’s words will be repeated again and our laptops might start to truly open the book “where men may read strange matters’.