New article: Visual narrative structure

March 25, 2013 12:19 am by Neil Cohn

I am happy to announce I have a new paper published in the journal Cognitive Science that outlines my theory of sequential image understanding. I argue that approaches like “panel transitions” cannot account for the creation of meaning in sequential images, and I offer a new theory of “Visual Narrative Grammar” that better accounts for how we comprehend visual narratives.

You can download this study along with my other papers, or directly to the pdf here.

This new theory uses narrative categories similar to traditional notions of narrative (though operationalized) that are organized into hierarchic constituents. The basics of this theory are outlined, along with diagnostic methods for testing these categories and constituents. Finally, I outline how this theory can apply beyond the sequential images found in comics to the understanding of film and verbal discourse.

This paper presents the basics of this theory for the first time in a cohesive paper, though this paper actually only consists of part of the broader theory of Visual Narrative Grammar. I hope to discuss the theory in full in subsequent papers and books. I should note also that my experiments on the cognition of understanding comics (also available for download) use this theory as their basis.

Here’s the full abstract:

Narratives are an integral part of human expression. In the graphic form, they range from cave paintings to Egyptian hieroglyphics, from the Bayeux Tapestry to modern day comic books (Kunzle, 1973; McCloud, 1993). Yet not much research has addressed the structure and comprehension of narrative images, for example, how do people create meaning out of sequential images? This piece helps fill the gap by presenting a theory of Narrative Grammar. We describe the basic narrative categories and their relationship to a canonical narrative arc, followed by a discussion of complex structures that extend beyond the canonical schema. This demands that the canonical arc be reconsidered as a generative schema whereby any narrative category can be expanded into a node in a tree structure. Narrative “pacing” is interpreted as a reflection of various patterns of this embedding: conjunction, left-branching trees, center-embedded constituencies, and others. Following this, diagnostic methods are proposed for testing narrative categories and constituency. Finally, we outline the applicability of this theory beyond sequential images, such as to film and verbal discourse, and compare this theory with previous approaches to narrative and discourse.

Cohn, Neil. (2013). Visual Narrative Structure. Cognitive Science, 37 (3), 413-452 DOI: 10.1111/cogs.12016