Högre seminarium: Aleksandra Ćwiek, Nicolaus Copernicus University in Toruń, Polen

Abstract (engelska)

The body as an orchestra: The natural statistics of uni- and multimodal utterances.

Human communication engages multiple parts of the body, each with distinct biomechanical properties, yet all functioning within a unified physical framework.

In this talk, I examine how vocal and gestural signals interact when produced simultaneously, as opposed to separately. The analysis is based on a substantial referential communication dataset, where 142 participants conveyed 84 concepts using gesture alone, nonlinguistic vocalization alone, or a combination of both modalities.

The results first demonstrate that concepts vary systematically in how readily they can be expressed through different communicative modalities. Participants' self-assessments of expressibility closely align with both actual communicative success and the efficiency of information transfer. I then analyze the physical properties of the signals, comparing acoustic, kinematic, and kinetic features across unimodal and multimodal productions.

The preliminary findings reveal modality-specific adaptations: when both modalities are used together, vocal features tend to become more prominent, while gestural features are reduced. These results suggest that multimodal communication involves not merely the addition of independent channels, but rather the coordinated reorganization of an integrated bodily system.

Seminariet hålls på engelska och tolkas till svenskt teckenspråk.