I’m very excited to announce that I have a new paper out in the latest issue of Neuropsychologia on how the brain understands sequences of images, “The grammar of visual narratives: Neural evidence for constituent structure in visual narrative comprehension.” This was actually one of my dissertation projects, so I’m very happy to finally have it published.
In this paper, I provide evidence that sequential images are processed by creating “constituents”— groupings of panels—rather than just “transitions” between the meanings of panels. Furthermore, disrupting these groupings in the “narrative grammar” of sequential images evokes the same brain responses as when you violate the grammar of sentences.
You can download the full paper here (direct link to pdf). I’ve also made a video summarizing the paper:
Official abstract:
Constituent structure has long been established as a central feature of human language. Analogous to how syntax organizes words in sentences, a narrative grammar organizes sequential images into hierarchic constituents. Here we show that the brain draws upon this constituent structure to comprehend wordless visual narratives. We recorded neural responses as participants viewed sequences of visual images (comics strips) in which blank images either disrupted individual narrative constituents or fell at natural constituent boundaries. A disruption of either the first or the second narrative constituent produced a left-lateralized anterior negativity effect between 500 and 700 ms. Disruption of the second constituent also elicited a posteriorly-distributed positivity (P600) effect. These neural responses are similar to those associated with structural violations in language and music. These findings provide evidence that comprehenders use a narrative structure to comprehend visual sequences and that the brain engages similar neurocognitive mechanisms to build structure across multiple domains.
Cohn, Neil., Jackendoff, Ray., Holcomb, Phillip. J., & Kuperberg, Gina. R. (2014). The grammar of visual narrative: Neural evidence for constituent structure in sequential image comprehension. Neuropsychologia, 64, 63-70. doi: 10.1016/j.neuropsychologia.2014.09.018
Comments
Hey Neil,
I've been coming to your site off and on for a while because your studies interest me as I am a filmmaker and am always interested in ways to improve my craft.
I'll have to admit thought that this video for some reason kinda went over my head.
I feel that it is very smaller in working towards an understanding of why certain editing techniques work and other don't.
Would that be a fair comparison?
If so, how do you feel i can understand these findings better and equally as important how can i use that understanding to write better "visual sentences" in film?
thanks so much!
KIKO!!!
kiko_pictures@yahoo.com