My research explores the structure and cognition of drawings and visual narratives, like comics. In particular, my work argues that drawings and sequential images are structured and processed using similar principles and mechanisms as spoken and signed languages. My work explores these "visual languages" across all levels of structure. I use linguistic theory to detail how these systems are structured, and corpus analysis to examine how they differ across cultures. Finally, I use psychological methods of experimentation, primarily cognitive neuroscience (EEG), to examine how people's brains process individual and sequential images, and how graphics are integrated with text in multimodal relationships. I am currently writing and drawing a non-fiction graphic book about this research about the relationship between graphic communication and language, and the multimodal connections between them.
My teaching focuses primarily on my areas of expertise within visual and multimodal communication. I teach the Visual Language bachelors course designed around my linguistic theories about visual communication. My masters courses further zoom in on areas of this research to focus on how we make meaning of text and image combined (Multimodal Communication with Joost Schilperoord) and how narratives are structured (Cognition of Visual Narratives). I also teach the research skills courses on EEG research, and on Visual Thinking and Composition, which teaches students basic drawing and visual communication skills.
I maintain several collaborations across the world focusing on a variety of aspects of research on visual and multimodal communication. These include research on attention and eye-movements, multimodality, autism, prediction, second language learning, and education. I am open to collaborations for most all aspects of visual language research. For more information, see my website.