My research on visual language at its core is about how do we communicate using pictures and words? My consulting work has directly used my research from linguistics and cognitive psychology to contribute towards more effective communication.
I’m particularly interested to work on projects related to education, progressive causes (such as environmentalism), and science. However, I am generally open-minded, and if you think I can provide a unique point of view to your ideas, projects, or inventions, I’d be happy to help make that happen. Here’s a few projects that I’m particularly proud of…
I have been writing and researching about emoji for almost 10 years, including doing cognitive research on the structure and comprehension of their vocabulary, grammar, and how they combine with written language. I have consulted for various researchers and companies related to emoji, such as helping Discord understand how their users think about emoji. I’m most proud of my work with Jennifer Daniel at Google and Unicode in the creation of new emoji. Together, we have proposed several face emoji, drawing on our research on the symbology of comics and manga. Many of our emoji are now in keyboards around the world like the breath face, melting face, dotted-line, and holding back tears emoij.
I have long been an advocate of using comics in education, and even created comics-essays for my classes in college. I have been consulting on comics and education for over 15 years, including for Archie Comics and educational companies like CAST, Inc, where I created comics about photosynthesis to be tested for use in classrooms, and for TrueFiktion, who are develop apps for teaching history through comics.
I have also served as a consultant for LingoZING!, which is an app that uses comics in an engaging interface to teach languages. Finally, I am a member of the Research Committee for Reading with Pictures, a non-profit devoted to the promotion of comics in education.
Comics have been shown to be an effective tool for communicating complex concepts. Because of this, companies have turned to using comics in order to report the news. I first consulted on this topic for Microsoft FUSE Labs who were creating a system to automatically generate news comics compiled from text and photos from Twitter.
More recently, I partnered with BBC News Labs to help create their system of automatically generating news comics. This system directly implements my theories of visual language by fusing a flexible library of images with journalists’ text to create an automatically generated news comic. To learn more, check out my Youtube video describing the creation of the system.