Cracking the body-language code to read emotions

Emily Underwood / ScienceNOW /

On the reality television show “Extreme Makeover: Home Edition,” the lucky recipient gets a first look at his newly renovated home. For a split second, his face contorts with — shock? Joy? During intense emotional experiences, there’s a fleeting moment when expressions of pleasure and pain are hard to distinguish. In fact, others read intense emotion more effectively by looking at a person’s body language than by watching their facial expressions, a new study suggests.

Most studies of facial cues rely on a set of stylized, recognizable expressions — perhaps made by actors in photographs. The actors make expressions meant to be obvious enough to translate across cultures: anger, disgust, fear, joy, sadness and surprise. But these stylized images don’t necessarily reflect the expressions that people make in the real world, says Hillel Aviezer, a neuropsychologist at who is now at The Hebrew University of Jerusalem and lead author of the new study, published online Thursday in Science. Moreover, when emotions get particularly extreme, people undergoing fleeting peaks of intense pain, joy, grief, or anger look surprisingly similar, Aviezer says. From the face, at least, “when you compare extreme pain to extreme pleasure, you really can’t tell them apart,” he says.

And yet most people are rarely confused about whether someone is experiencing grief or joy. To figure out what tips us off, Aviezer and his colleagues showed photos of professional tennis players to 45 Princeton University students, randomly divided into three groups of 15. Each tennis player had just won or lost an important match, and the participants rated the players’ contorted facial expressions from negative to positive on a scale from 1 to 9, with 5 marking the neutral midway point. One group of participants looked at head-to-toe photos of the players, the second group looked at only the players’ bodies, and the third group looked at only their heads. Only the final group had trouble making the correct identification, suggesting that facial expressions alone didn’t tell them whether the players were joyous or in despair.

With the help of photo-editing software, the team then switched the winning player’s heads with those of losers. To keep participants from noticing the trick, they shuffled the doctored photos with similar images. Participants still labeled winners or losers according to the players’ posture, not their facial expressions. In interviews conducted after the study, the researchers learned that cues such as whether a hand was open or clenched were more important than facial cues in interpreting expressions. Yet, in a separate experiment that asked 20 participants to state whether they would use body language, facial expressions, or both to evaluate emotion, 80 percent believed that they could judge the full-body photos by facial expression alone, Aviezer says. That result underscores our bias toward faces and how little we credit body language, he says.

To see if bodily gestures were more expressive in other contexts, the researchers performed a similar experiment with photographs of people in other high-intensity situations: crying at funerals, winning extravagant prizes on reality TV shows, getting their nipples and ears pierced, and having orgasms. Again, without body language to provide context, viewers struggled to correctly read facial expressions. In fact, they rated isolated faces displaying positive emotions more negatively than faces displaying negative emotions.