Visual Language Research

Research on visual language covers a wide range of topics, just like the study of spoken and signed languages. At the present, my main research questions are:


  • Visual Narrative - How do we make sense of sequential images? What principles and processing allow us to comprehend the individual images? How do people understand narratives and events? Is there a fluency to sequential image understanding? How does the brain do all this?


  • Cross-cultural visual languages - How do the drawing systems and sequential images of cultures around the world differ? Do these differences tie into deeper aspects of cognition?

  • Cognition of drawing - What is the cognitive structure of drawing? Can this structure inform us about the ways people learn to draw? What does it tell us about the treatment of drawings and images in culture?

  • Cognitive processes across domains - What aspects of cognition are shared by different human behaviors, such as spoken, visual, and signed languages? How do these domains overlap with the cognition of music or event understanding?

These questions extend across the domains of structure, cognition, and cross-cultural comparison, and thereby demand a multi-pronged approach to research. Since visual language is a natural part of human cognition similar to spoken or signed languages, we can study it by drawing on the research and methods of linguistics, psychology, and cognitive neuroscience.  Broadly, these methods include:

Theory

Developing theories on the types of knowledge our minds use to make sense of drawings and visual language. These theories provide the foundation for...

Experiments

Psychology experiments examining the comprehension of drawings and sequential images both testing behavior (reaction times, ratings, etc.) and directly measuring processing in the brain (EEG, fMRI, etc.).

Crosscultural

Comparative analysis of visual languages from across the world by coding the properties within comics, leading to cross-cultural comparisons.


Interested in Research?


If you’re interested in reading academic research on visual language, please go to my Downloadable Papers or check out my books. The "introductory writings" page has more casual and basic writing on visual language theory, while The Visual Linguist blog has more casual and exploratory writing on this topic, as well as updates about ongoing research. You can also find video lectures and podcasts describing this research.

For people interested in participating in studies, check out our links to any online studies we might currently be running.


Donations





Collaboration and Research Opportunities


For those interested in doing research, I also provide several resources providing advice and pointing towards research materials.

If you are currently a student in San Diego County and interested in helping with research, please contact me directly.

I am always interested in pursing research with willing collaborators. If you are currently doing research and/or are interested in doing visual language research and would like me to collaborate and/or advise, please contact me. My current collaborators are:

  • Marta Kutas, UC San Diego
  • Jeff Elman, UC San Diego
  • John Drury, Stony Brook University
  • Tom Foulsham, University of Essex
  • David Wagner, University of Stavanger-Norway
  • Michael Arbib, University of Southern California