Stockholm university
Denna sida på svenska

Johanna MeschProfessor

About me

Johanna Mesch [johɑnɑ mɜʃ] 
Professor
Sign Language Section
Department of Linguistics, Stockholm University

Johanna Mesch
Presentation in International Sign Language.

 

 

Research

My research activities:

One of my areas of expertise is tactile sign language communication. Tactile sign language is used by people with deafblindness, whose primary language is sign. The interlocutors receive signs in tactile-kinaesthetic mode through the hands.

The second area of expertise is sign language corpora. In 2003, I was involved in sign language corpora for the first time, and I carried out a pilot project in digital humanities that established an online sign language archive for an initiative called European Cultural Heritage Online (ECHO) at the Max Planck Institute for the History of Science, Berlin. The sizeable national project “Korpus för det svenska teckenspråket”, the Swedish Sign Language Corpus, funded by Riksbankens Jubileumsfond (2009-2011), has been completed, but work on the corpus data will continue for several years. In addition to the L1 corpus in Swedish Sign Language, we have built a learner corpus in STS with Krister Schönström since 2013 and during the RJ-financed research project 2017–2019. I was a visiting professor at Universidade Federal de Santa Catarina in Florianópolis, Brazil, from August to October 2018. My research interests include tactile sign language, sign language corpora, dictionaries, and language technology (language-centred AI).

I am also involved in the following research networks:

Finished project/network:

  • Edusign & Signedu, Erasmus+ KA220-HED, March 2022-September 2024 https://edusignedu.com/en/ 
  • MADS project, collaborations between seven higher education institutions with a view of setting up a new joint Master programme in Applied Deaf Studies (MADS), Nov 2022-March 2024
    https://signs.hw.ac.uk/projects/mads/
  • Nordic Signed Languages Corpora Network, 2022-2023
  • NIICTE, an international multidisciplinary network of higher education professionals working on inclusion and inclusive communication in tertiary education, 2023
  • A Clarin Resource Family for Sign Languages, Dec 2021-May 2022, financed by CLARIN ERIC https://www.clarin.eu/resource-families
  • Från tal till tecken - att lära sig Svenskt teckenspråk som andraspråk, TATE-projektet, 2017-2019
  • Språkdiversitet, SU

Research projects

Edusign & Signedu

Edusign & Signedu is a European collaborative project that aims to increase society's awareness of deaf-related issues and create opportunities for more countries to train sign language interpreters.

Swedish Sign Language Corpus

The Swedish Sign Language Corpus is a database for the Swedish sign language. The corpus is produced by sign language speakers to facilitate teaching and research on sign language.

Publications

A selection from Stockholm University publication database

  • Extended Interlingual Index for the Project's Core Sign Languages and Languages covered in Work Package 9

    2024. Sam Bigeard (et al.).

    Report

    The purpose of the interlingual index is to link the lexical resources of the sign languages of the project. This is the final project release of the index, which covers both the core sign languages of the project as well as additional sign languages.

    Read more about Extended Interlingual Index for the Project's Core Sign Languages and Languages covered in Work Package 9

  • Swedish Sign Language resources from a user’s perspective

    2024. Johanna Mesch (et al.). Proceedings of the LREC-COLING 2024 11th Workshop on the Representation and Processing of Sign Languages, 54-61

    Conference

    The Swedish Sign Language Dictionary [Svenskt teckenspråkslexikon] is one of the most visited websites at Stockholm University, with four million visits each year. The dictionary is an easy-to-use resource for the community, families, relatives, students, educators, researchers and other stakeholders that can be accessed through the website, app, and mobile platforms. STS-korpus is an online interface for the Swedish Sign Language Corpus that is linked to the STS Dictionary, enhancing its utility. Other applications, like TSP Quiz and the STS transcription tool, will also be evaluated. In January 2024, we conducted a survey to explore how users utilise Swedish sign language resources in their everyday lives, studies and work, regardless of hearing status and sign language skills. The purpose is to evaluate these resources from a user’s perspective, including aspects such as user-friendliness, relevance, comprehensibility and effectiveness in aiding language learning or communication.

    Read more about Swedish Sign Language resources from a user’s perspective

  • Exploring latent sign language representations with isolated signs, sentences and in-the-wild data

    2024. Fredrik Malmberg (et al.). Proceedings of the LREC-COLING 2024 11th Workshop on the Representation and Processing of Sign Languages, 378-383

    Conference

    Unsupervised representation learning offers a promising way of utilising large unannotated sign language resources found on the Internet. In this paper, a VQ-VAE model is trained to learn a codebook of motion primitives from sign language data. For training, we use isolated signs and sentences from a sign language dictionary. Three models are trained: one on isolated signs, one on sentences, and one mixed model. We test these models by comparing how well they are able to reconstruct held-out data from the dictionary, as well as an in-the-wild dataset based on sign language videos from YouTube. These data are characterized by less formal and more expressive signing than the dictionary items. Results show that the isolated sign model yields considerably higher reconstruction loss for the YouTube dataset, while the sentence model performs the best on this data. Further, an analysis of codebook usage reveals that the set of codes used by isolated signs and sentences differ significantly. In order to further understand the different character of the datasets, we carry out an analysis of the velocity profiles, which reveals that signing data in-the-wild has much higher average velocity than dictionary signs and phrases. We believe these differences also explain the large differences in reconstruction loss observed

    Read more about Exploring latent sign language representations with isolated signs, sentences and in-the-wild data

  • Content questions in sign language from theory to language description via corpus, experiments, and fieldwork

    2024. Robert Gavrilescu, Carlo Geraci, Johanna Mesch. Proceedings of the LREC-COLING 2024 11th Workshop on the Representation and Processing of Sign Languages, 298-306

    Conference

    The theory of language structure informs us about what we should expect when we want to investigate a certain construction. However, reality is often richer than what theories predict. In this study, we start from a theoretically informed set of hypotheses about the structure of wh-questions in sign language, we test them using a sign language corpus, a designed production experiment, and structured fieldwork in three sign languages, Swedish, Greek and French Sign Languages. The results will inform us on what type of contribution each research method can provide to reach accurate language descriptions.

    Read more about Content questions in sign language from theory to language description via corpus, experiments, and fieldwork

  • Creating a multifaceted corpus of Swedish Sign Language

    2023. Johanna Mesch. Advances in Sign Language Corpus Linguistics, 242-261

    Chapter

    This chapter discusses the preparatory work of creating a collection of corpora, together functioning as a multifaceted corpus of the same sign language, across three data subsets of signing in different modalities and on different learning levels, namely visual signing (deaf/hard-of-hearing/CODA), tactile signing (deafblind), and L2 signing (hearing second-language learners). This work, led by native signers and based on personal experiences in the research field, involved planning and adapting data collection and annotation of the Swedish Sign Language corpora over time to ensure a corpus infrastructure that is uniform across data sets, compatible with a parallel lexical database, and available as a searchable resource for the public.

    Read more about Creating a multifaceted corpus of Swedish Sign Language

  • Meaning-making in tactile cross-signing context

    2023. Johanna Mesch, Eli Raanes. Journal of Pragmatics 205, 137-150

    Article

    Cross-linguistic studies of tactile sign language are still not widely performed internationally. For this study, four deafblind informants in two different tactile sign languages, Swedish Sign Language and Norwegian Sign Language, participated in the recording at a social and cultural workshop where they worked together to create a mutual understanding in their conversations. The study provides new information on how tactile and bodily signals are incorporated in dialogues where the speakers are not familiar with each other's signing. The results illuminate various tactile communicative strategies used in negotiating in cross-signing dialogues. By the selected analyzed examples, this study contributes to knowledge of how language and interaction skills are brought into the process of understanding each other, despite linguistic barriers.

    Read more about Meaning-making in tactile cross-signing context

  • Self‐repair in hearing L2 learners’ spontaneous signing: A developmental study

    2023. Johanna Mesch, Krister Schönström. Language learning 73 (1), 136-163

    Article

    This study presents a corpus-based investigation of self-repairs in hearing adult L2 (M2L2, second modality and second language) learners of Swedish Sign Language (Svenskt teckenspråk, STS). This study analyses M2L2 learners’ STS conversations with a deaf signer and examines the learners’ self-repair practices and whether there are differences among learners of different proficiency levels. This provides a description of characteristics of self-repair made by M2L2 learners as well as the frequency and distribution of self-repair categories. The results show that the frequency of self-repair decreases with increased proficiency, at least after the initial stage. Furthermore, the self-initiated repair categories of the beginners are often phonological repairs, while intermediate learners tend to carry out self-repairs at the lexical and syntactic level. The results also reveal a specific type of STS repair linked to fingerspelling repairs. We discuss the effects of second modality learning as well as the relationship between monitoring and language proficiency.

    Read more about Self‐repair in hearing L2 learners’ spontaneous signing

  • Teckenspråkslexikografi – utmaningar i en annan modalitet

    2023. Johanna Mesch (et al.). Nordiska studier i lexikografi 16, 225-240

    Conference

    Swedish Sign Language Dictionary, first published in 2008, was updated to a new version in May 2022. This version has improved search functions and user interface. The lexical database has over 24,000 unique signs and 6,600 example sentences and is being continuously updated with new signs and examples. Each entry contains rich information in the form of video and photos demonstrating the sign, as well as in text and sign transcription describing the execution of the sign. For many signs, there is additional information, for instance, on the use of the sign in context or on the sign’s origin. The different search paths that have been developed are based on the structure of the signs and on different subject areas. Swedish Sign Language Dictionary is connected to the Swedish Sign Language Corpus, which currently contains approximately 190,000 occurrences of signs. This collaboration takes place via the STS-korpus, a web-based tool for the  presentation of signs in natural language use. The collaboration between these two language resources is also concerned with lexicographic issues.

    Read more about Teckenspråkslexikografi – utmaningar i en annan modalitet

  • A cross-linguistic comparison of reference across five signed languages

    2022. Lindsay Ferrara (et al.). Linguistic typology

    Article

    Do signers of different signed languages establish and maintain referencethe same way? Here we compare how signers of five Western deaf signedlanguages coordinate fully conventionalized forms with more richly improvisedsemiotics to identify and talk about referents of varying agency. The five languages(based on a convenience sample) are Auslan, Irish Sign Language, Finnish SignLanguage, Norwegian Sign Language, and Swedish Sign Language. Using tenretellings of Frog, Where Are You? from each language, we analyze tokens ofreferring expressions with respect to: (a) activation status (new vs. maintained vs.re-introduced); (b) semiotic strategy (e.g., pointing sign, fingerspelling, enactment);and (c) animacy (human vs. animal vs. inanimate object). Statisticalanalysis reveals many similarities and some differences across the languages. Forexample, signers of each language typically used conventionalized forms toidentify new referents, and less conventional strategies to maintain and reintroducereferents. Differences were mainly observed in relation to the patterning across animacy and activation categories and in the use of fingerspelled wordsfrom ambient spoken/written languages. We suggest that doing reference in thesesigned languages involves both signed language-specific and ecology-specificstrategies. The latter may be attributed to the different social and historical trajectoriesof each language.

    Read more about A cross-linguistic comparison of reference across five signed languages

  • Use and acquisition of mouth actions in L2 sign language learners

    2021. Johanna Mesch, Krister Schönström. Sign Language and Linguistics 4 (1)

    Article

    This article deals with L2 acquisition of a sign language, examining in particular the use and acquisition of non-manual mouth actions performed by L2 learners of Swedish Sign Language. Based on longitudinal data from an L2 learner corpus, we describe the distribution, frequency, and spreading patterns of mouth actions in sixteen L2 learners at two time points. The data are compared with nine signers of an L1 control group.

    The results reveal some differences in the use of mouth actions between the groups. The results are specifically related to the category of mouthing borrowed from spoken Swedish. L2 signers show an increased use of mouthing compared to L1 signers. Conversely, L1 signers exhibit an increased use of reduced mouthing compared with L2 signers. We also observe an increase of adverbial mouth gestures within the L2 group. The results are discussed in relation to previous findings, and within the framework of cross-linguistic influence.

    Read more about Use and acquisition of mouth actions in L2 sign language learners

  • Mouthings in Swedish Sign Language

    2021. Johanna Mesch, Krister Schönström, Sebastian Embacher. Grazer Linguistische Studien 93, 107-135

    Article

    This paper deals with the non-manual mouth actions of Swedish Sign Language, Svenskt teckenspråk (STS). Based on data from the Swedish Sign Language Corpus and the Swedish Sign Language as L2 Corpus, we compare the use of mouthings in deaf L1 as well as hearing L2 signers. The use, distribution and frequency of mouthings are explored and described quantitatively and qualitatively. The results reveal some similarities as well as differences in the use of mouthings between the groups. Furthermore, the analysis reveals qualitative differences related to the properties of mouthings i.e. full and reduced mouthings among L1 as well as L2 learners of STS. Challenges of the analysis of mouthings will be discussed.

    Read more about Mouthings in Swedish Sign Language

  • Conveying environmental information to deafblind people: a study of tactile sign language interpreting

    2020. Sílvia Gabarró-López, Johanna Mesch. Frontiers in Education 5

    Article

    Many deafblind people use tactile sign language and interpreters in their daily lives. Because of their hearing and sight status, the role of interpreters does not only involve translating the content expressed by other deaf or hearing people, but it also involves conveying environmental information (i.e., multimodal communication regarding what is happening at a given moment to be able to understand the context). This paper aims to contribute to the field of tactile sign language interpreting by describing how two Tactile Swedish Sign Language interpreters convey environmental information to two deafblind women in a particular situation, that is, a guided visit to a cathedral by a hearing Norwegian speaker. We expect to find various strategies including the use of haptic signs (i.e., a system of signs articulated on the body of the deafblind person aimed to provide environmental and interactional information). After summarizing the small amount of existing research on the issue to date, we present our data and how they were annotated. Our analysis shows that a variety of strategies are used, including Tactile Swedish Sign Language, using locative points to show locations with some type of contact with the body of deafblind individuals, depicting shapes on the palm of the hand of deafblind individuals, using objects to depict shapes, touching elements of the cathedral with the hands or with the feet such as surfaces, and walking around. Some of these strategies are more frequent than others and some strategies are also used in combination, whereas others are used in isolation. We did not observe any use of haptic signs to convey environmental information in our data, which calls for further research on which criteria apply to use this strategy in a particular situation.

    Read more about Conveying environmental information to deafblind people

Show all publications by Johanna Mesch at Stockholm University

Edit the profile