Johanna Mesch [johɑnɑ mɜʃ]
Scientific head of the Sign Language Section
Department of Linguistics, Stockholm University
My research activities:
One of my areas of expertise is tactile sign language communication. Tactile sign language is used by people with deafblindness, whose primary language is sign. The interlocutors receive signs in tactile-kinaesthetic mode through the hands.
The second area of expertise is sign language corpora. In 2003, I was involved in sign language corpora for the first time, I carried out a pilot project in digital humanities that established an online sign language archive for an initiative called European Cultural Heritage Online (ECHO) at the Max Planck Institute for the History of Science, Berlin. The large national project “Korpus för det svenska teckenspråket”, the Swedish Sign Language Corpus, funded by Riksbankens Jubileumsfond (2009-2011), has been completed, but work on the corpus data will continue for several years. In addition to the L1 corpus in Swedish Sign Language, we have also been building a learner corpus in STS with Krister Schönström since 2013 and during the RJ financed research project 2017–2019. I served as a visiting professor at Universidade Federal de Santa Catarina in Florianópolis, Brazil, in August-October 2018.
I am also involved in the following research networks:
- Swe-Clarin https://sweclarin.se/
- Språkdiversitet https://www.ling.su.se/langdiv
- A Clarin Resource Family for Sign Languages, Dec 2021-May 2022, financed by CLARIN ERIC https://www.clarin.eu/resource-families
- Nordic Signed Language Corpus Network https://r.jyu.fi/DWC
- DEPICT - How do interpreters use depiction when interpreting for deafblind individuals?, projektet av Høgskulen på Vestlandet, 2022-2025 https://www.hvl.no/en/research/prosjekt/depict/wp/
- SMILE-II about multimodal sign language technology for sign language learning and assessment 2021-2024 https://data.snf.ch/grants/grant/193686
- Teckenspråkskorpus, Signbank, teckenspråkslexikon (Swedish Sign Language Corpus and Dictionary project), continuing
We will organize a series of three workshops which will bring together a network of Nordic signed language researchers, who all work with multimodal SL corpora using a cognitive-functional linguistic approach. The workshops will develop empirical corpus methods for SLs and contribute to the planning of a corpus-exploiting international research project that will focus on similarities and differences between Swedish, Finnish and Norwegian SL.
A selection from Stockholm University publication database
Use and acquisition of mouth actions in L2 sign language learners
2021. Johanna Mesch, Krister Schönström. Sign Language and Linguistics 4 (1)Article
This article deals with L2 acquisition of a sign language, examining in particular the use and acquisition of non-manual mouth actions performed by L2 learners of Swedish Sign Language. Based on longitudinal data from an L2 learner corpus, we describe the distribution, frequency, and spreading patterns of mouth actions in sixteen L2 learners at two time points. The data are compared with nine signers of an L1 control group.
The results reveal some differences in the use of mouth actions between the groups. The results are specifically related to the category of mouthing borrowed from spoken Swedish. L2 signers show an increased use of mouthing compared to L1 signers. Conversely, L1 signers exhibit an increased use of reduced mouthing compared with L2 signers. We also observe an increase of adverbial mouth gestures within the L2 group. The results are discussed in relation to previous findings, and within the framework of cross-linguistic influence.
Mouthings in Swedish Sign Language
2021. Johanna Mesch, Krister Schönström, Sebastian Embacher. Grazer Linguistische Studien 93, 107-135Article
This paper deals with the non-manual mouth actions of Swedish Sign Language, Svenskt teckenspråk (STS). Based on data from the Swedish Sign Language Corpus and the Swedish Sign Language as L2 Corpus, we compare the use of mouthings in deaf L1 as well as hearing L2 signers. The use, distribution and frequency of mouthings are explored and described quantitatively and qualitatively. The results reveal some similarities as well as differences in the use of mouthings between the groups. Furthermore, the analysis reveals qualitative differences related to the properties of mouthings i.e. full and reduced mouthings among L1 as well as L2 learners of STS. Challenges of the analysis of mouthings will be discussed.
Annoteringskonventioner för teckenspråkstexter
2021. Johanna Mesch, Lars Wallin.Report
Conveying environmental information to deafblind people
2020. Sílvia Gabarró-López, Johanna Mesch. Frontiers in Education 5, 1-12Article
Many deafblind people use tactile sign language and interpreters in their daily lives. Because of their hearing and sight status, the role of interpreters does not only involve translating the content expressed by other deaf or hearing people, but it also involves conveying environmental information (i.e., multimodal communication regarding what is happening at a given moment to be able to understand the context). This paper aims to contribute to the field of tactile sign language interpreting by describing how two Tactile Swedish Sign Language interpreters convey environmental information to two deafblind women in a particular situation, that is, a guided visit to a cathedral by a hearing Norwegian speaker. We expect to find various strategies including the use of haptic signs (i.e., a system of signs articulated on the body of the deafblind person aimed to provide environmental and interactional information). After summarizing the small amount of existing research on the issue to date, we present our data and how they were annotated. Our analysis shows that a variety of strategies are used, including Tactile Swedish Sign Language, using locative points to show locations with some type of contact with the body of deafblind individuals, depicting shapes on the palm of the hand of deafblind individuals, using objects to depict shapes, touching elements of the cathedral with the hands or with the feet such as surfaces, and walking around. Some of these strategies are more frequent than others and some strategies are also used in combination, whereas others are used in isolation. We did not observe any use of haptic signs to convey environmental information in our data, which calls for further research on which criteria apply to use this strategy in a particular situation.
2020. Zrajm Öqvist, Nikolaus Riemer Kankkonen, Johanna Mesch. Proceedings of the 9th Workshop on the Representation and Processing of Sign Languages, 177-180Conference
In this paper we describe STS-korpus, a web corpus tool for Swedish Sign Language (STS) which we have built during the past year, and which is now publicly available on the internet. STS-korpus uses the data of Swedish Sign Language Corpus (SSLC) and is primarily intended for teachers and students of sign language. As such it is created to be simple and user-friendly with no download or setup required. The user interface allows for searching – with search results displayed as a simple concordance – and viewing of videos with annotations. Each annotation also provides additional data and links to the corresponding entry in the online Swedish Sign Language Dictionary. We describe the corpus, its appearance and search syntax, as well as more advanced features like access control and dynamic content. Finally we say a word or two about the role we hope it will play in the classroom, and something about the development process and the software used. STS-korpus is available here: https://teckensprakskorpus.su.se
Transitivity prominence within and across modalities
2019. Carl Börstell (et al.). Open Linguistics 5 (1), 666-689Article
We investigate transitivity prominence of verbs across signed and spoken languages, based on data from both valency dictionaries and corpora. Our methodology relies on the assumption that dictionary data and corpus-based measures of transitivity are comparable, and we find evidence in support of this through the direct comparison of these two types of data across several spoken languages. For the signed modality, we measure the transitivity prominence of verbs in five sign languages based on corpus data and compare the results to the transitivity prominence hierarchy for spoken languages reported in Haspelmath (2015). For each sign language, we create a hierarchy for 12 verb meanings based on the proportion of overt direct objects per verb meaning. We use these hierarchies to calculate correlations between languages – both signed and spoken – and find positive correlations between transitivity hierarchies. Additional findings of this study include the observation that locative arguments seem to behave differently than direct objects judging by our measures of transitivity, and that relatedness among sign languages does not straightforwardly imply similarityin transitivity hierarchies. We conclude that our findings provide support for a modality-independent, semantic basis of transitivity.
The Use of Signing Space in Signed News Broadcasts / L’utilisation de l’espace de signation dans les émissions signées
2019. Pia Simper-Allen, Johanna Mesch. Lidil (60)Article
Cet article étudie l’utilisation des tokens dans deux types de journaux télévisés, l’un pour un public sourd adulte et l’autre pour un public sourd jeune, en langue des signes suédoise. Un token est un point vide et non topographique dans l’espace de signation qui se situe devant le présentateur. Notre échantillon contient 1084 tokens qui ont été placés à un point précis de cet espace de signation pour faire référence au concept introduit par chaque token au cours du discours. Les présentateurs exploitent ce mécanisme de référence fréquemment et les types de signes les plus utilisés à ce propos sont des signes lexicaux, des pointages et des verbes directionnels. La plupart des tokens sont placés dans l’espace de signation gauche ou droit du présentateur, tandis que l’espace de signation frontal est moins utilisé. Le nombre de tokens est plus réduit dans l’introduction et la conclusion des informations télévisées. Nous pensons que ces résultats pourraient être des spécificités des programmes d’information en langue des signes. Dans notre analyse, nous avons aussi tenu compte de l’utilisation des images à l’écran et de l’effet de celles-ci sur la création des tokens.
The uses of corpora in L1 and L2/Ln sign language pedagogy
2019. Lorraine Leeson (et al.). The Routledge Handbook of Sign Language Pedagogy, 339-352Chapter
This chapter explores the use of sign language corpora in L1 and L2/Ln sign language classes. We discuss how corpora have been developed and used by linguists working on spoken and, more recently, signed languages. The corpora can be leveraged for pedagogic purposes. Examples from corpora-based pedagogical practice in Sweden, Ireland, and Australia are offered. We outline some possible future pedagogical applications of sign language corpora and propose some research pathways that presently remain unexplored.