As we've seen in a recent story ("Customer Service Bots Are Getting Better at Detecting Your Agitation"), facial recognition software has moved beyond matching faces to trying to infer the emotional state of the face. At the heart of this effort is the assumption that, generally, facial expressions convey the same emotional state across cultures. Recent research shows this might not be the case.
In the 1960s, psychologist Paul Ekman came up with the method that has become the standard way to test this: present a collection of pictures of Westerners with different facial expressions to people living in isolated cultures and ask them what emotion was being conveyed. His research showed universality in understanding facial expressions across cultures. This has become an accepted axiom of this field ever since. However, in 2011, psychologists Carlos Crivelli and José-Miguel Fernández-Dols investigated the assumptions and methodology of the Ekman experiments. They traveled to the Trobriand Islands off the coast of Papua New Guinea and performed their own experiment using pictures of facial expressions.
Crivelli found that they matched smiling with happiness almost every time. Results for the other combinations were mixed, though. For example, the Trobrianders just couldn’t widely agree on which emotion a scowling face corresponded with. Some said this and some said that. It was the same with the nose-scrunching, pouting, and a neutral expression. There was one facial expression, though, that many of them did agree on: a wide-eyed, lips-parted gasping face (similar to above [link]) that Western cultures almost universally associate with fear and submission. The Trobrianders said it looked “angry.”
The work is being well received in the field, such as by social psychologist Alan Fridlund who noted that the researchers did an excellent job immersing themselves in the Trobriander culture before conducting the experiment.
Despite agreeing broadly with the study’s conclusions, Fridlund doubts it will sway hardliners convinced that emotions bubble forth from a common font. Ekman’s school of thought, for example, arose in the post–World War II era when people were seeking ideas that reinforced our common humanity, Fridlund says. “I think it will not change people’s minds. People have very deep reasons for adhering to either universality or cultural diversity.”
An abstract is available: The fear gasping face as a threat display in a Melanesian society.
[How might this affect Unicode's emoticons (i.e. code points starting at \U0001F600)? -Ed.]
Related Stories
SRI International, the Silicon Valley research lab where Apple's virtual assistant Siri was born, is working on a new generation of virtual assistants that respond to users' emotions.
As artificial-intelligence systems such as those from Amazon, Google, and Facebook increasingly pervade our lives, there is an ever greater need for the machines to understand not only the words we speak, but what we mean as well—and emotional cues can be valuable here (see "AI's Language Problem").
"[Humans] change our behavior in reaction to how whoever we are talking to is feeling or what we think they're thinking," says William Mark, who leads SRI International's Information and Computing Sciences Division. "We want systems to be able to do the same thing."
[...] The system is designed to identify emotional state based on a variety of cues, including typing patterns, speech tone, facial expressions, and body movements.
SenSay could, for example, add intelligence to a pharmacy phone assistant. It might be able to tell from a patient's pattern of speech if he or she were becoming confused, then slow down.
The machine-learning-based technology is trained on different scenarios, depending on how it will be used. The new virtual assistants can also monitor for specific words that give away a person's mental state.
It works via text, over the phone, or in person. If someone pauses as he or she types, it could indicate confusion. In person, the system uses a camera and computer vision to pick up on facial characteristics, gaze direction, body position, gestures, and other physical signals of how a person is feeling.
(Score: 2) by t-3 on Saturday October 29 2016, @05:32PM
I wuld have said surprise. Submission is much more of a body language thing, as well as fear, although that would show in the eyes too. It would be nice if we could see the actual pictures used though.
(Score: 2) by RamiK on Saturday October 29 2016, @05:54PM
Submission was likely a mistake in reference to Plutchik's wheel of emotions seeing how that "deer caught in the headlights" look covers awe and adjacent almost entirely.
compiling...
(Score: 2) by Arik on Saturday October 29 2016, @05:55PM
"I wuld have said surprise."
Startled, alerted, something like that yeah.
"Submission is much more of a body language thing"
Precisely. If that face is accompanied by a hands-up posture, that would be submission. But if that same face was accompanied motions toward weapons, or with weapons, it would be very different.
The 'wide eyes' indicate fear or surprise but those things do not necessarily lead to submission - they may very well trigger aggression instead.
It also seems that while this can be an immediate and involuntary response to surprise, there's a slower semi-voluntary form of it as well, where you will see someones eyes go wide not in an instant, but over a longer period, a good fraction of a second, and while they may not be consciously aware of exactly what they are doing THAT is not a purely involuntary response either, you're watching them consciously classify a threat, or consciously broadcast to those around them that classification.
If laughter is the best medicine, who are the best doctors?
(Score: 2) by wonkey_monkey on Saturday October 29 2016, @05:45PM
I'm shocked to learn of this.
:)
systemd is Roko's Basilisk
(Score: 2) by AthanasiusKircher on Saturday October 29 2016, @09:04PM
I'm shocked to learn of this.
:)
Hmm, well I guess expressions really aren't universal. I'd have thought if you were shocked, you have said:
:-O
or at least something like:
O_O
(Score: 2) by wonkey_monkey on Saturday October 29 2016, @11:17PM
http://s2.quickmeme.com/img/00/00ddce11eb42d72fd22ab4b7d9875eea4419fd39916dbe8dd946dfe8558ec180.jpg [quickmeme.com]
systemd is Roko's Basilisk
(Score: 2) by AthanasiusKircher on Sunday October 30 2016, @02:53AM
Thought it might be, actually. But then I thought you might just be using :) as a generic sarcasm sign, as some people do, rather than a face referent (which it seems some people forget "smileys" even are).
Anyhow, sorry for trying to make a joke off of a misreading of your joke.
(Score: 2) by wonkey_monkey on Sunday October 30 2016, @11:09AM
But then I thought you might just be using :) as a generic sarcasm sign, as some people do
Who are these animals?
systemd is Roko's Basilisk
(Score: 1, Funny) by Anonymous Coward on Saturday October 29 2016, @06:00PM
I am shocked - SHOCKED, I say - that this hideous display of naked racism was permitted to stain the innocent pages of the internet!
Next thing, these researchers will be suggesting that cultures, and even sub-cultures, may tend to push people in the direction of certain negative proclivities, instead of celebrating the wealth and variety of human experience!
They might even suggest that some of this cultural drift may tend to genetic drift, and propose that human brains may differ to measurable extent in their range of capacity, as per genetic heritage.
They probably vote for Trump. Revoke their science licences!
(Score: 2) by esperto123 on Sunday October 30 2016, @01:01PM
If that is their angry face, it amazes me they were not wiped out by enemies a long time ago.
Kidding asside, if you find this kind of small difference only in isolated small communities, I think it is easier explained by that specific culture changing and inate behavior than that behavior not being inate at all, besides, people born blind do the same faces as non-blind people when happy, scared, angry, etc. this CANNOT be learned.