News and events Tilburg University

Neil Cohn: emoji on scientific basis help people to communicate better

Published: 18th October 2021 Last updated: 18th October 2021

The melting face emoji, the dotted-line emoji, the holding-back tears emoji (coming soon), the breath-face emoji, 😮💨: all used daily worldwide in interaction on social media, were co-created by a scientist from Tilburg School of Humanities and Digital Sciences. Neil Cohn Ph.D. was interviewed last month by the New York Times on his work designing a new emoji, the one with a melting face. Why does a researcher work on emoji? How can science add value to an emoji and what is the impact?

On his website Cohn writes that he has been fascinated with drawing and comics his whole life. By age 10, he was selling his own created comics via mail order catalogue, and as a teenager in the 1990s he worked for Image Comics and Todd McFarlane Productions at Comic-Con International. His artistic work has been published in several books.

Assistant Professor Neil Cohn

While taking a linguistics class in college, Neil began noticing similarities between the structure of language and the structure found in the "visual language" used in comics. Since then, his research has argued that the structure, cognition, and development of this visual language is similar to that of spoken and signed languages.

How does a scientist get involved with what could be the symbol of modern popular culture, the emoji?

‘It was by accident, I did research on visual language and comics and a former student of mine was invited to write an article about emoji. He came to me and interviewed me. After that other journalists started asking for more information. So I wrote an article for BBC, and publicity became even more. No research had been done yet on the things I was talking about, so I fit it into my research program of visuals and graphics.

There was for instance no systematic analysis on the origins of emoji symbolism, which originally came from Japanese comics, or manga. From the manga come emoji like the shaded top hat, the hearts in eyes, the streaming tears. Although they originate from Japan, they have become an internationally understood visual vocabulary. Then I was contacted by Jennifer Daniel, who works for Google, asking about my expertise and they requested that I help to propose new emoji based on my studies of comics. The face emoji is most used, so I tried to fill in gaps there.

Unique research on emoji

One of my ideas was to do a dotted line emoji to show introversion, depression, etc. because dotted lines mean ‘invisibility,’ but they were afraid people would think the emoji was broken and proposed we made it faded instead. So, I made an experiment and started a survey to check if people would understand: look at this emoji, what does it mean? Many reactions were consistent. Apparently it was unique to do actual research for an emoji proposal!’

emojis

Of course the emoji makes just a small part of his research. In 2020, Neil began the project Visual narratives as a window into language and cognition (TINTIN) funded by a European Research Council Starting Grant. This project has created software tools to facilitate examining the cross-cultural differences between the visual languages used in comics of the world, and to explore their relationships to the structures in spoken languages.

We should not underestimate the importance of emoji, says Cohn. ‘Creating pictures makes us human beings, no others species does that. Emoji are visuals that add to words to enrich how we communicate. Pure text is impoverished, and lacks physical expressions like gestures or the sound of speech. Writing takes away that extra non-verbal information, but emoji may allow us to re-incorporate that type of expression into our text. To help create new emoji is gratifying, because it has societal impact. I can contribute to helping people communicate in an easier and richer way, to express themselves better.’