PICTREE Project

The PICTREE Project

Pictures are often thought to be fairly simple compared to language. Where language is known to use patterns in a vocabulary and grammar, pictures are often thought to reflect what we see, and as a reflection of vision would not have complex structure of their own. In contrast, my work has broadly shown that pictures are also built of patterns, including both a vocabulary and a grammar. This “PICTREE Project” targets one aspect of pictorial communication that has been claimed to be unique to language: hierarchic structure.

The PICTREE Project—officially called “Hierarchic structure in pictorial communication”—aims to investigate how the brain encodes hierarchic information about individual and sequential images. The project is a five year project funded by a European Research Council (ERC) Consolidator grant to Neil Cohn.

The PICTREE project has three primary themes where we are measuring people’s brainwaves to investigate hierarchic structure. These include the primary aspects of individual and sequential pictures, along with their relations to other modalities:

  1. Prajwal Prakash is studying individual images for how graphics are constructed, while postdoc Irmak Hacımusaoğlu is studying hierarchy in visual morphology.
  2. Maryam Torabi is studying the narrative grammar of sequential images with complex patterns that involve “distance dependencies”
  3. Lenneke Lichtenberg is studying whether the similar brain responses that appear to manipulations of “grammars” in sentences, music, or visual narratives share a common origin in the brain

Aspects of visual narrative comprehension are explored at length in the book Who Understands Comics?, and some of our prior work set the stage for these projects: