New paper: Structural complexity in visual narratives

June 1, 2019 3:05 pm by Neil Cohn

2019 so far has been a flurry of published papers for me, and here’s yet another. My paper “Structural complexity in visual narratives: Theory, brains, and cross-cultural diversity” is now published in the book collection Narrative Complexity and Media: Experiential and Cognitive Interfaces. The book is an extensive resource (468 pages!) including many chapters about the cognitive study of narrative. Mine is one of several that discusses visual narratives, along with complementary chapters by Joe Magliano and James Cutting. So, the book is highly recommended!

In this paper, I tackle the issue of “narrative complexity” in three ways. First, I describe the way in which sequences of images are built in terms of their underlying structure. This complexity comes from the narrative structure, and how various schematic principles combine to create patterns with “complexity” in their architecture similar to what is found in syntactic structure in sentences.

The second level of complexity comes in how these narrative patterns manifest in different types of comics from around the world. We coded the properties of various comics to see how comics from Europe, the United States, and Asia might differ in their narrative patterns. We found that they indeed vary, with comics from Asia (Japan, Korea, Hong Kong) using more complex sequencing patterns than those from Europe or the United States. This is important because such diversity is systematic, implying that they are encoded in the minds of their authors and readers.

The third level of complexity comes in how visual narratives like comics are processed. Many theories posit that we understand comics by simply linking meanings between panels. This implies a fairly uniform process guided only by updating meaning from image to image. However, neurocognitive research implies that the brain actually uses several interacting mechanisms in the processing of narrative image sequences, balancing both meaning and a narrative structure of the type described in the previous sections.

Altogether, this paper outlines a balance between theoretical, cross-cultural, and neurocognitive research that identifies complexity at multiple levels.

The paper is available in the book itself, but a downloadable preprint version is available here or on my downloadable papers page.

Cohn, Neil. 2019. Structural complexity in visual narratives: Theory, brains, and cross-cultural diversity. In Grishakova, Marina and Maria Poulaki (Ed.). Narrative Complexity and Media: Experiential and Cognitive Interfaces  (pp. 174-199). Lincoln: University of Nebraska Press