Visual Language research galore!

April 30, 2014 3:00 pm by Neil Cohn

Can I brag for a moment? Not about me, really, but about my students!

This quarter I’m teaching a course on the “Cognition of Comics” for the UCSD Cognitive Science Department. While the classes do discuss experiments and studies that have been done related to how people understand comics, and how they differ across cultures, I wanted this class to give students the opportunity to do science rather than just learn about it.

So, the course actually revolves around students doing their own research projects related to topics in visual language. While I’ve tried to help guide them towards interesting projects, they’ve mostly come up with them on their own, and boy do they seem exciting!

We have 7 experiments that people are doing, investigating the cognition of “upfixes” (symbols that go above characters heads), how people use systematic representations in drawings, how people navigate page layouts, and even how color might affect emotion in comics.

We also have projects coding across many comics to see what the properties of various structures are. Before this class, there were less than ten studies doing this sort of corpus analysis on the structure of comics. There was a dissertation by Neff, McCloud’s study of panel transitions, a few done by Charles Forceville, and two done by me. In this class alone, there are roughly 14 corpus studies!

The comparisons of populations range from looking at how various structural aspects of American Visual Language change over time in superhero comics, how structures might be similar or different between genres of Japanese manga (shoujo, shonen, sports, etc.), how structures might differ between comics done by English vs. French vs. Spanish speakers, and differences between American, Japanese, and/or Chinese comics. (among others)

These comparisons are looking at how panels frame information, the constraints on how manga use super-deformation, how “visual morphemes” and symbols are used across comics of different cultures, how schematic graphic information (like the way people draw eyes or hair) are systematically used across different authors, how page layouts are structured, and how text and images interact with each other. Plus, quite a few other topics!

Hopefully, many of these papers will find information that will be significant enough to publish, suddenly increasing what we know about the structure of different comics across the world by over 200%! It’s going to be a very exciting quarter!