Investigating Acoustic Correlates of Whisper Scoring for L2 Speech Using Forced Alignment with the Italian Component of the ISLE corpus
Étude des corrélats acoustiques du scoring de la parole d'apprenant à l'aide d'un alignement forcé avec la partie italienne du corpus ISLE
Résumé
Automatic Speech Recognition (ASR) can be used to analyse L2 speech but researchers cannot be sure that the ASR transcriptions accurately represent learner speech. We aim to confront the ASR outputs with the acoustic analysis of learner speech. Whisper (Radford, 2023) provides transcriptions and probabilities associated with the predicted transcriptions. This paper analyses how global phonetic analyses of learner data can be used to potentially confirm these Whisper probability scores assigned to learner transcriptions. We tested the Italian component of the ISLE corpus with phonetic analyses of 23 learners of English. We compared the levels assigned to these speakers by the corpus experts to the outputs of Whisper's tiny model. We discuss the phonetic features that may account for these Whisper predictions using acoustic data extracted from forced alignment. We try to correlate the levels assigned to the speakers in the ISLE corpus with the quality of the phonetic realisation, using global vocalic measurements such as the convex hull or Euclidian distances between monophthongs. We show that Levenshtein distance to the reference transcription of the Whisper tiny model (measured using Levenshtein distance to the read text) correlates with the grades assigned by the annotators.
Cette étude évalue la congruence des scores de probabilités associés aux prédictions du LLM Whisper avec les métadonnées du corpus ISLE (niveau CECRL), la distance de Levenshtein des transcriptions et les corrélats acoustiques extraits d'un alignement forcé.
Origine | Fichiers éditeurs autorisés sur une archive ouverte |
---|