This page features downloadable academic papers on visual language research. Also, see my introductory writings for more accessible writing for laypeople, and the Visual Linguist blog for more casual and exploratory writing, along with updates about ongoing research. For more detailed information about visual language and its study, check out my book, The Visual Language of Comics.
- PDF – open pdfs or links in new window.
- NEW – recently added papers.
- *** – recommended introductory papers.
Papers are organized by their major topics as best as possible. Because aspects of visual language are interconnected, some papers may discuss multiple topics. For a chronological listing of papers, see my vitae.
The study of visual language covers a wide range of topics, just like that of spoken or signed languages. These papers discuss the broader relationship between visual language, comics, and lingusitics, and how to study them.
NEW *** Cohn, Neil. 2018. Visual Language Theory and the scientific study of comics. In Wildfeuer, Janina, Alexander Dunst, Jochen Laubrock (Ed.). Empirical Comics Research: Digital, Multimodal, and Cognitive Methods. (pp. 305-328) London: Routledge. (PDF)
*** Cohn, Neil. 2016. Sequential images are not universal, or Caveats for using visual narratives in experimental tasks. In Papafragou, A., Grodner, D., Mirman, D., & Trueswell, J.C. (Eds.) (2016). Proceedings of the 38th Annual Conference of the Cognitive Science Society. Austin, TX: Cognitive Science Society. (PDF)
*** Cohn, Neil. 2015. Climbing trees and seeing stars: Combinatorial structure in comics and diverse domains. In Toivonen, Ida, Piroska Csúri & Emilie van der Zee. (Ed). Structures in the Mind: Essays on Language, Music, and Cognition in Honor of Ray Jackendoff (pp. 379-392). Cambridge: MIT Press. (PDF)
Cohn, Neil. 2014. Building a better “comic theory”: Shortcomings of theoretical research on comics how to overcome them. Studies in Comics. 5(1): 57-75. (PDF)
Cohn, Neil. 2012. Comics, linguistics, and visual language: The past and future of a field. In Bramlett, Frank (ed). Linguistics and the Study of Comics (pp. 92-118). New York: Palgrave Macmillan. (PDF)
Cohn, Neil. 2010. Japanese Visual Language: The structure of manga. In Manga: An Anthology of Global and Cultural Perspectives, edited by T. Johnson-Woods (pp. 187-203). New York: Continuum Books. (PDF)
Cohn, Neil. 2005. Un-defining "comics." International Journal of Comic Art. 7(2): 236-248. (PDF)
My main project in visual langauge research has always been exploring how sequences of images create meaning in the mind and brain. I am primarily interested in how such competence is structured, how it is processed in the brain, and how that system relates to other cognitive domains like verbal discourse and narrative, event structure, and even musical cognition. Because the theory of narrative grammar is fairly complex, students and researchers interested in the diagnostics and procedures of implementing it should read this tutorial on How to analyze visual narrative (PDF).
Cohn, Neil. 2018. In defense of a “grammar” in the visual language of comics. Journal of Pragmatics. 127: 1-19. (PDF)
Cohn, Neil. 2015. How to analyze visual narrative. Visual Language Lab Resource (PDF)
Cohn, Neil. 2015. Narrative conjunction’s junction function: The interface of narrative grammar and semantics in sequential images. Journal of Pragmatics 88: 105-132. (PDF)
*** Cohn, Neil. 2014. The architecture of visual narrative comprehension: the interaction of narrative structure and page layout in understanding comics. Frontiers in Psychology. 5:680. (Online version & PDF, Japanese PDF)
Cohn, Neil. 2013. Visual narrative structure. Cognitive Science 37(3): 413-452. (PDF)
Cohn, Neil. 2010. The limits of time and transitions: Challenges to theories of sequential image comprehension. Studies in Comics 1(1): 127-147. (PDF)
Experimental and corpus papers
NEW Coderre, Emily L., Neil Cohn, Sally K. Slipher, Mariya Chernenok, Kerry Ledoux, and Barry Gordon. 2018. Visual and linguistic narrative comprehension in autism spectrum disorders: Neural evidence for modality-independent impairments. Brain and Language. 186: 44-59. (PDF)
Cohn, Neil, Martin Paczynski, and Marta Kutas. 2017. Not so secret agents: Event-related potentials to semantic roles in visual event comprehension. Brain and Cognition. 119: 1-9. (PDF, Poster summary)
Cohn, Neil and Marta Kutas. 2017. What’s your neural function, visual narrative conjunction? Grammar, meaning, and fluency in sequential image processing. Cognitive Research: Principles and Implications. 2(27): 1-13. (PDF, Online version, Poster summary)
Cohn, Neil and Patrick Bender. 2017. Drawing the line between constituent structure and coherence relations in visual narratives. Journal of Experimental Psychology: Learning, Memory, and Cognition. 43(2): 289-301. (PDF)
Cohn, Neil. 2016. From Visual Narrative Grammar to Filmic Narrative Grammar: The narrative structure of static and moving images. In Wildfeuer, Janina & John A. Bateman (eds.). Film text analysis: New perspectives on the analysis of filmic meaning. (pp. 94-117). London: Routledge. (PDF)
Foulsham, Tom, Dean Wybrow, and Neil Cohn. 2016. Reading without words: Eye-movements in the comprehension of comic strips. Applied Cognitive Psychology. 30: 566-579 (PDF)
Hagmann, Carl Erick, and Neil Cohn. 2016. The pieces fit: Constituent structure and global coherence of visual narrative in RSVP. Acta Psychologica. 164:157-164. (PDF)
Cohn, Neil, and Marta Kutas. 2015. Getting a cue before getting a clue: Event-related potentials to inference in visual narrative comprehension. Neuropsychologia 77: 267-278. (PDF)
Cohn, Neil, and Eva Wittenberg. 2015. Action starring narratives and events: Structure and inference in visual narrative comprehension. Journal of Cognitive Psychology. 27(7): 812-828. (PDF)
Cohn, Neil, Ray Jackendoff, Phillip Holcomb, and Gina Kuperberg. 2014. The grammar of visual narratives: Neural evidence for constituent structure in visual narrative comprehension. Neuropsychologia. 64: 63-70. (PDF, Video summary)
Cohn, Neil. 2014. You’re a good structure, Charlie Brown: The distribution of narrative categories in comic strips. Cognitive Science, 38(7): 1317-1359. (PDF)
Cohn, Neil, and Martin Paczynski. 2013. Prediction, events, and the advantage of Agents: The processing of semantic roles in visual narrative. Cognitive Psychology. 67 (3):73-97. (PDF)
Cohn, Neil, Martin Paczynski, Ray Jackendoff, Phillip Holcomb, and Gina Kuperberg. 2012. (Pea)nuts and bolts of visual narrative: Structure and meaning in sequential image comprehension. Cognitive Psychology 65(1): 1-38. (PDF, Poster version)
Cohn, Neil, Amaro Taylor-Weiner, and Suzanne Grossman. 2012. Framing attention in Japanese and American comics: Cross-cultural differences in attentional structure. Frontiers in Cultural Psychology. 3: 1-12. (PDF)
Cohn, Neil. 2011. A different kind of cultural frame: An analysis of panels in American comics and Japanese manga. Image [&] Narrative 12 (1): 120-134. (PDF)
Visual languages rarely appear in isolation, and usually accompany written language. These papers explore these types of multimodal interactions.
NEW Cohn, Neil, Tim Roijackers, Robin Schaap, and Jan Engelen. 2018. Are emoji a poor substitute for words? Sentence processing with emoji substitutions. In T.T. Rogers, M. Rau, X. Zhu, & C. W. Kalish (Eds.), Proceedings of the 40th Annual Conference of the Cognitive Science Society. (pp. 1524-1529) Austin, TX: Cognitive Science Society. (PDF, Poster version)
NEW Manfredi, Mirella, Neil Cohn, Mariana De Araújo Andreoli, and Paulo Sergio Boggio. 2018. Listening beyond seeing: Event-related potentials to audiovisual processing in visual narrative. Brain and Language.185: 1-8. (PDF)
Cohn, Neil, Ryan Taylor, and Kaitlin Pederson. 2017. A picture is worth more words over time: Multimodality and narrative structure across eight decades of American superhero comics. Multimodal Communication. 6(1): 19-37. (PDF, Video Presentation)
Manfredi, Mirella, Neil Cohn, and Marta Kutas. 2017. When a hit sounds like a kiss: an electrophysiological exploration of semantic processing in visual narrative. Brain and Language. 169: 28-38. (PDF)
Pratha, Nimish K., Natalie Avunjian, and Neil Cohn. 2016. Pow, punch, pika, and chu: The structure of sound effects in genres of American comics and Japanese manga. Multimodal Communication. 5(2): 93-109. (PDF, Video Presentation)
*** Cohn, Neil. 2016. A multimodal parallel architecture: A cognitive framework for multimodal interactions. Cognition 146: 304-323. (PDF)
These papers discuss aspects of comic page layouts—their "External Compositional Structure."
NEW Cohn, Neil, Jessika Axnér, Michaela Diercks, Rebecca Yeh, and Kaitlin Pederson. 2017. The cultural pages of comics: Cross-cultural variation in page layouts. Journal of Graphic Novels and Comics. (PDF)
Pederson, Kaitlin and Neil Cohn. 2016. The changing pages of comics: Page layouts across eight decades of American superhero comics. Studies in Comics. 7(1): 7-28. (PDF, Video Presentation)
Cohn, Neil and Hannah Campbell. 2015. Navigating comics II: Constraints on the reading order of page layouts. Applied Cognitive Psychology. 29: 193-199. (PDF)
Cohn, Neil. 2013. Navigating comics: An empirical and theoretical approach to strategies of reading comic page layouts. Frontiers in Cognitive Science. 4: 1-15. (Online version & PDF)
These papers discuss the cognition of drawing: how we create meaning by making graphic marks. They explore how drawing is structured in the mind/brain, how this ability is developed, and the effect of culture on these abilities.
Cohn, Neil. 2014. Framing “I can’t draw”: The influence of cultural frames on the development of drawing. Culture & Psychology. 20(1): 102-117. (PDF)
*** Cohn, Neil. 2012. Explaining "I can't draw": Parallels between the structure and development of language and drawing. Human Development 55(4): 167-192. (PDF)
These papers cover aspects of visual language "vocabulary"—the meaningful elements of images.
Cohn, Neil. Combinatorial morphology in visual languages. In Booij, Geert (Ed.). The Construction of Words: Advances in Construction Morphology. (pp. 175-199). London: Springer. (PDF)
Cohn, Neil, Vivian Wong, Kaitlin Pederson & Ryan Taylor. (2017). Path salience in motion events from verbal and visual languages. In G. Gunzelmann, A. Howes, T. Tenbrink, & E. J. Davelaar (Eds.), Proceedings of the 39th Annual Conference of the Cognitive Science Society (pp. 1794-1799). Austin, TX: Cognitive Science Society. (PDF, Poster PDF)
Cohn, Neil, Beena Murthy, and Tom Foulsham. 2016. Meaning above the head: Combinatorial constraints on the visual vocabulary of comics. Journal of Cognitive Psychology. 28(5): 559-574. (PDF)
Cohn, Neil and Sean Ehly. 2016. The vocabulary of manga: Visual morphology in dialects of Japanese Visual Language. Journal of Pragmatics. 92: 17-29. (PDF, Listing of visual morphemes in JVL)
Cohn, Neil, and Stephen Maher. 2015. The notion of the motion: The neurocognition of motion lines in visual narratives. Brain Research. 1601: 73-84. (PDF, Poster version).
Cohn, Neil. 2013. Beyond speech balloons and thought bubbles: The integration of text and image. Semiotica. 2013(197): 35-63. (PDF)
Cohn, Neil. 2010. Extra! Extra! Semantics in comics!: The conceptual structure of Chicago Tribune advertisements. Journal of Pragmatics 42 (11): 3138–3146. (PDF)
Cohn, Neil. 2007. A visual lexicon. Public Journal of Semiotics. 1(1): 35-56. (PDF)