Potential Field “Multimodal Communication”

Communication is inherently multimodal. In spoken languages, visual and other modalities complement speech through manual and facial gestures. Sign languages—languages with a fully grammaticalized communication system in the visual domain—can also use multiple channels of expression simultaneously, including manual signs, facial expressions, body posture, and head movements. Written language often integrates visual elements such as emojis or images, and animal communication also exhibits multimodality, for example when vocal and manual cues are combined.

Within theoretical linguistics, multimodality is a relatively recent field of research that seeks to integrate such phenomena into formal linguistic theory. A central goal is to capture the formal components of visual communication by identifying meaning-bearing elements within the continuous signals of body, head, arm, and hand movements. A key challenge is the treatment of iconic and depictive elements—vocal or visual expressions whose form reflects aspects of their meaning, such as speech-accompanying gestures—which have long been difficult to integrate into existing formal models.

The Potential Field Multimodal Communication aims to strengthen multimodality research at Goethe University and within the Rhine–Main Universities (RMU) by creating spaces for structured exchange across disciplinary boundaries. It is deliberately open and exploratory in scope, aiming to bring researchers into conversation whose work engages with multimodality in different ways; for example in linguistics, literary studies, theatre and performance studies, anthropology, film and media studies, cultural studies, history, philosophy, psychology and neurocognition, musicology, computer science, graphic illustration, and related fields.