Research project Syntax and the body: the structuring of language in multimodal sense-making
This project investigates how syntax interfaces with gestures, gaze, and bodily movements in authentic conversations. Using video data from several languages, we explore how the body and language interact to create, negotiate, and understand meaning in interaction.
When we talk, we rely not only on words and grammar, but also on gaze, gestures, facial expressions, posture, and subtle features of the voice. This international project investigates how syntax actually works in everyday interaction through detailed analysis of video-recorded conversations in Swedish, Estonian, French, Japanese, Mandarin, and German.
We examine how turns are built step by step through the coordination of speech and bodily action – for example, how a glance can signal that a turn is coming to an end, how a pointing gesture clarifies what a word refers to, or how a pause and a hesitation sound shape the interpretation of an utterance. By comparing languages, we identify both language-specific solutions and recurring cross-linguistic patterns. The project thus offers a more realistic view of grammar as something living, dynamic, and deeply embedded in human interaction.
This 6-year research environment challenges traditional rule-based views of syntax by exploring the ways in which syntax is grounded in embodied interaction across languages and cultures.
Building on the intriguing idea of “linguistic bodies” (Di Paolo et al. 2018), the project places interacting bodies at the center of human sense-making. The aim is to examine whether syntax is but one part of communication, whose systematic properties are best understood in relation to how people coordinate bodily actions in interaction.
By combining methods from ethnomethodological Conversation Analysis, Interactional Linguistics, and functional grammar, the project looks for regular patterns between participants’ bodily positions and movements and the types of syntactic structures they use. It draws on video-recorded interactions in six typologically diverse languages.
We address the following research questions:
- How are syntactic structures and bodily conduct coordinated to perform social actions? Is there evidence for routinization of syntactic-bodily practices (across languages)?
- In what ways are such practices contingent upon the local interactional context? How are boundary phenomena between linguistic and bodily expressions, such as nonlexical vocalizations, involved in these formats?
- How is the sensing and moving body implicated in producing syntax and prosody?
- What can a multimodal analysis contribute to our understanding of the structuring of syntactic constructions and their function in projecting and completing speaker turns?
The project develops an empirically based theory of embodied syntax, reveals cross-linguistic differences, and advances dialogic, functional accounts of language, with implications for language education and digital interaction design.
The project is funded by a Research Environment Grant from the Swedish Research Council (no. 2024-01863).