Can face classifiers make a reliable inference on criminality?
November 23, 2016
by Nancy Owano, Tech Xplore
(Tech Xplore)—Researchers have worked on a new program that they say can pick up criminality just by analyzing facial features.
One thought that may come to mind is watching some old comedy where an angry adult scolds a young boy and says, "I can tell just by looking at you that you're no good!" Your reaction might be to laugh.
Two Chinese researchers, however, are quite serious about their work. They have authored a paper about machine learning and they examine how a computer may be able to tell who is a non-criminal and who is.
Xiaolin Wu and Xi Zhang, Chinese researchers from Shanghai Jiao Tong University, wrote a paper which is on arXiv. Their paper is titled "Automated Inference on Criminality using Face Images."
But really? How can algorithms do any better in an exercise where you actually examine the face for inference on criminality?
Katyanna Quach in The Register said, "It's true that machines don't have emotions or conscience to be considered subjective, but that doesn't mean data can't be biased." Ben Sullivan in Motherboard pointed out that the researchers maintained that the data sets were controlled for race, gender, age and facial expressions.
The authors stated, "We are the first to study automated face-induced inference on criminality free of any biases of subjective judgments of human observers. By extensive experiments and vigorous cross validations, we have demonstrated that via supervised machine learning, data-driven face classifiers are able to make reliable inference on criminality."
Sullivan wrote more about the question of bias: "Wu told Motherboard that human bias didn't come into it. 'In fact, we got our first batch of results a year ago. We went through very rigorous checking of our data sets, and also ran many tests searching for counterexamples but failed to find any,' said Wu."
How they tested: Xiaolin Wu and Xi Zhang fed into a machine learning algorithm facial images of 1,856 people. Nearly half were convicted criminals.
Motherboard said they used standard ID photographs (not mugshots) of Chinese males between the ages of 18 and 55. The men did not have facial hair,
("We stress that the criminal face images in Sc are normal ID photos not police mugshots," wrote the researchers.)
MIT Technology Review picked up on their methods: "They then used 90 percent of these images to train a convolutional neural network to recognize the difference and then tested the neural net on the remaining 10 percent of the images." Results? They said that the classifiers performed "consistently well" and produced "evidence for the validity of automated face-induced inference on criminality."
MIT Technology Review's discussion of their findings in "Emerging Technology from the arXiv," said that the pair found that "the neural network could correctly identify criminals and noncriminals with an accuracy of 89.5 percent."
What kinds of features had bearing? The researchers said they found "some discriminating structural features for predicting criminality, such as lip curvature, eye inner corner distance, and the so-called nose-mouth angle."
Looking at their findings, Quach said, "It's bad news for those who have smaller mouths, curvier upper lips and closer-set eyes, as you look more like a crook, apparently. On average, criminals have a 19.6 per cent smaller nose-mouth angle, a larger upper lip curvature at 23.4 per cent, and a 5.6 per cent shorter distance between the inner corners of the eyes."
A remark in Hacker News: "I thought this was a joke when I read the abstract, but it appears to be a genuine paper."
And if you still think the very idea of looking at facial features to determine anything of the sort is a bit of a stretch you would not be alone. In 2016, the paper's focus concerned some people who would prefer to avoid any concept that suggests using physical features to determine criminality.
Writing in The Intercept, Sam Biddle said that "no computer or software is created in a vacuum. Software is designed by people, and people who set out to infer criminality from facial features are not free from inherent bias."
MIT Technology Review, looking ahead, said, "Of course, this work needs to be set on a much stronger footing. It needs to be reproduced with different ages, sexes, ethnicities, and so on. And on much larger data sets." Also, said the report, "All this heralds a new era of anthropometry, criminal or otherwise," and there is room for more research "as machines become more capable."
More information: Automated Inference on Criminality using Face Images, arXiv:1611.04135 [cs.CV] arxiv.org/abs/1611.04135
We study, for the first time, automated inference on criminality based solely on still face images. Via supervised machine learning, we build four classifiers (logistic regression, KNN, SVM, CNN) using facial images of 1856 real persons controlled for race, gender, age and facial expressions, nearly half of whom were convicted criminals, for discriminating between criminals and non-criminals. All four classifiers perform consistently well and produce evidence for the validity of automated face-induced inference on criminality, despite the historical controversy surrounding the topic. Also, we find some discriminating structural features for predicting criminality, such as lip curvature, eye inner corner distance, and the so-called nose-mouth angle. Above all, the most important discovery of this research is that criminal and non-criminal face images populate two quite distinctive manifolds. The variation among criminal faces is significantly greater than that of the non-criminal faces. The two manifolds consisting of criminal and non-criminal faces appear to be concentric, with the non-criminal manifold lying in the kernel with a smaller span, exhibiting a law of normality for faces of non-criminals. In other words, the faces of general law-biding public have a greater degree of resemblance compared with the faces of criminals, or criminals have a higher degree of dissimilarity in facial appearance than normal people.
© 2016 Tech Xplore