Magical things happen at conferences sometimes. Back at the Cognitive Neuroscience Society conference in 2014, I ran into my graduate school friend, Carl Hagmann, who mentioned he was doing interesting work on rapid visual processing, where people are asked to detect certain objects within an image sequence that changes at really fast speeds (like 13 milliseconds). He noticed that I was doing things with image sequences too and thought we should try this rapid pace with visual narratives (similar to this old paper I blogged about).
Lo and behold, it actually happened, and now our paper is published in the journal Acta Psychologia!
Our paper examines how quickly people process visual narrative sequences by showing participants the images from comics at either 1 second or half a second. In some sequences, we flipped the order that images appeared. In general, we found that “switches” of greater distances were recognized with better accuracy and those sequences were rated as less comprehensible. Also, switching panels between groupings of panels were recognized better than those within groups, again showing further evidence that visual narratives group information into constituents.
This was quite the fun project to work on, and it marks a milestone: It’s the first “visual language” paper I’ve had published where I’m not the first author! Very happy about that, and there will be several more like it coming soon…
You can find the paper via direct link here (pdf) or on my downloadable papers page.
Abstract:
Recent research has shown that comprehension of visual narrative relies on the ordering and timing of sequential images. Here we tested if rapidly presented 6-image long visual sequences could be understood as coherent narratives. Half of the sequences were correctly ordered and half had two of the four internal panels switched. Participants reported whether the sequence was correctly ordered and rated its coherence. Accuracy in detecting a switch increased when panels were presented for 1 s rather than 0.5 s. Doubling the duration of the first panel did not affect results. When two switched panels were further apart, order was discriminated more accurately and coherence ratings were low, revealing that a strong local adjacency effect influenced order and coherence judgments. Switched panels at constituent boundaries or within constituents were most disruptive to order discrimination, indicating that the preservation of constituent structure is critical to visual narrative grammar.
Hagmann, Carl Erick, and Neil Cohn. 2016. “The pieces fit: Constituent structure and global coherence of visual narrative in RSVP.” Acta Psychologica 164:157-164. doi: 10.1016/j.actapsy.2016.01.011.
Comments