Investigating Acoustic Correlates of Whisper Scoring for L2 Speech Using Forced Alignment with the Italian Component of the ISLE corpus - Productions scientifiques du CLILLAC-ARP
Communication Dans Un Congrès Année : 2024

Investigating Acoustic Correlates of Whisper Scoring for L2 Speech Using Forced Alignment with the Italian Component of the ISLE corpus

Étude des corrélats acoustiques du scoring de la parole d'apprenant à l'aide d'un alignement forcé avec la partie italienne du corpus ISLE

Résumé

Automatic Speech Recognition (ASR) can be used to analyse L2 speech but researchers cannot be sure that the ASR transcriptions accurately represent learner speech. We aim to confront the ASR outputs with the acoustic analysis of learner speech. Whisper (Radford, 2023) provides transcriptions and probabilities associated with the predicted transcriptions. This paper analyses how global phonetic analyses of learner data can be used to potentially confirm these Whisper probability scores assigned to learner transcriptions. We tested the Italian component of the ISLE corpus with phonetic analyses of 23 learners of English. We compared the levels assigned to these speakers by the corpus experts to the outputs of Whisper's tiny model. We discuss the phonetic features that may account for these Whisper predictions using acoustic data extracted from forced alignment. We try to correlate the levels assigned to the speakers in the ISLE corpus with the quality of the phonetic realisation, using global vocalic measurements such as the convex hull or Euclidian distances between monophthongs. We show that Levenshtein distance to the reference transcription of the Whisper tiny model (measured using Levenshtein distance to the read text) correlates with the grades assigned by the annotators.
Cette étude évalue la congruence des scores de probabilités associés aux prédictions du LLM Whisper avec les métadonnées du corpus ISLE (niveau CECRL), la distance de Levenshtein des transcriptions et les corrélats acoustiques extraits d'un alignement forcé.
Fichier principal
Vignette du fichier
2024.nlp4call-1.2-1.pdf (981.41 Ko) Télécharger le fichier
Origine Fichiers éditeurs autorisés sur une archive ouverte

Dates et versions

hal-04855376 , version 1 (24-12-2024)

Licence

Identifiants

  • HAL Id : hal-04855376 , version 1

Citer

Nicolas Ballier, Adrien Méli. Investigating Acoustic Correlates of Whisper Scoring for L2 Speech Using Forced Alignment with the Italian Component of the ISLE corpus. 13th Workshop on Natural Language Processing for Computer Assisted Language Learning (NLP4CALL 2024), Université Rennes 2, France; University of Gothenburg, Sweden; Linköping University, Sweden, Oct 2024, Rennes, France. ⟨hal-04855376⟩
0 Consultations
0 Téléchargements

Partager

More