Beware, poker face: Automatic system spots micro-expressions

Humans have a tough enough time figuring out one another. We actually use two languages to communicate, through words and through facial expressions and sometimes they don't match.

What is tough is knowing what the other person is really feeling when such a mismatch takes place. ("I'm fine, don't bother. Just a little ache. I'm sure it will go away.")

Now that we have made advances in machine learning and AI, how are software engineers doing in coming up with programs that can look at a face and tell what the person is feeling?

There was recent news about Microsoft's solution for rating someone's face based on eight different emotion metrics.

In quick tests from Hot Hardware, they said they found the detection to be "very accurate," while CBC News reported that while "Some of the images we tested pulled a pretty accurate reading," they said others were "way off."

(A useful note here taken from MIT Technology Review is that "rapid developments in artificial intelligence in recent years have come about partly because of improved methods of computing. But these machines are useless without vast and accurate databases to train them.")

Now, a big step up in emotion-detection software deals with yet another language—the facial expression that is really a "micro-expression," real feelings that appear and disappear in the blink of an eye ("Oh, thank you, Auntie. A scented candle! How thoughtful." [Where am I going to put this crap. This is the third scented candle I have got and I don't like any of them.])

MIT Technology Review reports that scientists at the University of Oulu in Finland have both built and tested "the first machine vision system capable of spotting and recognizing micro-expressions and they say that it is already better than humans at the task."

The researchers have posted their paper on arXiv. "Reading Hidden Emotions: Spontaneous Micro-expression Spotting and Recognition" is by seven authors from the University of Oulu in Finland (X. Li, X. Hong, A. Moilanen, X. Huang, G. Zhao and M. Pietikainen) and from (T. Pfister) University of Oxford in the UK. The research was supported by the Academy of Finland, Tekes and Infotech Oulu.

How the authors defined micro-expression: "An ME is a very brief, involuntary FE which shows the emotion that people try to conceal. In contrast to ordinary FEs, MEs are very short (1/3 to 1/25 second), the precise length definition varies, and the intensities of involved muscle movements are subtle."

How they worked: Li and team created a database of videos showing micro-expressions in realistic conditions; this was not trivial because posed expressions don't cover the real-life scenario when the person is trying to hide an emotion at high stakes. As the authors stated, "In this work we focused on the study of spontaneous MEs, which are much more difficult to analyze than posed expressions explored in previous work. We proposed novel methods for both ME spotting and ME recognition."

They asked 20 individuals to watch a series of videos designed to invoke strong emotions. They were given a strong incentive to avoid showing emotion: they were told that that they would have to fill in a long, boring questionnaire explaining any emotions they did display.

Also, the team recorded on a high-speed camera at 100 frames per second. The team linked the emotions on display to the emotional content of the videos, said MIT Technology Review. This gave them a database with which to train their machine-learning algorithm.

The team used "a single frame showing the subject's face as a standard and comparing all subsequent frames against it to determine how the expression changed. Any change beyond a certain threshold was defined as a micro-expression, and these images set aside for further analysis," said the MIT Technology Review report.

Results? They make for interesting reading, said MIT Technology Review. "Li and co's machine matched human ability to spot and recognize microexpressions and significantly outperformed humans at the recognition task alone."

Of what practical value is their research? The authors wrote that "This method has many high-impact applications, particularly in lie detection, law enforcement and psychotherapy."

© 2015 Tech Xplore