We have carried out research on the psychology and linguistics of emoji, particularly in ongoing collaboration with Ben Weissman (Rensselaer Polytechnic Institute). This work has focused on how emoji convey meaning, and how they interact in multimodal interactions with written language. Our work has shown that people incorporate acquire the meanings of emoji with relative strengths. If emoji substitute for words in sentences, they are readily integrated into the grammar. However, emoji do not seem to be combined into sequences using a grammar of their own, and the way emoji combine into linear sequences is somewhat unnatural for graphic communication. Here’s some links to our research:

• Weissman, Benjamin, Neil Cohn, and Darren Tanner. The electrophysiology of lexical prediction of emoji and text. Neuropsychologia. 198 (108881):1-11 (Journal)
• Weissman, Ben, Jan Engelen, Elise Baas and Neil Cohn (2023), The Lexicon of Emoji? Conventionality Modulates Processing of Emoji. Cognitive Science, 47: e13275. (Journal)
• Cohn, Neil, Jan Engelen, and Joost Schilperoord. 2019. The grammar of emoji? Constraints on communicative pictorial sequencing. Cognitive Research: Principles and Implications. 4(1):33. (Online version & PDF)
• Cohn, Neil, Tim Roijackers, Robin Schaap, and Jan Engelen. 2018. Are emoji a poor substitute for words? Sentence processing with emoji substitutions. In T.T. Rogers, M. Rau, X. Zhu, & C. W. Kalish (Eds.), Proceedings of the 40th Annual Conference of the Cognitive Science Society. (pp. 1524-1529) Austin, TX: Cognitive Science Society. (PDF, Poster version)

Additional work is connected to emoji, such as the “upfixes” that float above characters heads like hearts or lightbulbs. This work has focused on the constraints on upfix structure, or how combining elements creates non-literal meanings:

• Cohn, Neil, Beena Murthy, and Tom Foulsham. 2016. Meaning above the head: Combinatorial constraints on the visual vocabulary of comics.  Journal of Cognitive Psychology. 28(5): 559-574. (PDF)