Can Artificial Intelligence Identify Reading Fluency and Level? Comparison of Human and Machine Performance


YILDIZ M., Keskin H. K., Oyucu S., Hartman D. K., Temur M., Aydoğmuş M.

Reading and Writing Quarterly, 2024 (SSCI) identifier

  • Yayın Türü: Makale / Tam Makale
  • Basım Tarihi: 2024
  • Doi Numarası: 10.1080/10573569.2024.2345593
  • Dergi Adı: Reading and Writing Quarterly
  • Derginin Tarandığı İndeksler: Social Sciences Citation Index (SSCI), Scopus, Academic Search Premier, Communication Abstracts, EBSCO Education Source, Educational research abstracts (ERA), ERIC (Education Resources Information Center), Linguistics & Language Behavior Abstracts, MLA - Modern Language Association Database, Psycinfo
  • Gazi Üniversitesi Adresli: Evet

Özet

This study examined whether an artificial intelligence-based automatic speech recognition system can accurately assess students’ reading fluency and reading level. Participants were 120 fourth-grade students attending public schools in Türkiye. Students read a grade-level text out loud while their voice was recorded. Two experts and the artificial intelligence-based automatic speech recognition system analyzed the recordings for reading errors. Following the analysis, a word error rate was calculated for both the experts and the artificial intelligence-based automatic speech recognition system. Word error rates were converted into reading accuracy rate scores. Inter-rater agreement and linear regression analyses were used to compare the raters’ reading fluency scores, and logistic regression analyses were used to compare the classification of readers according to their reading levels. Results showed that the difference between the scores of the artificial intelligence-based automatic speech recognition system and the expert scores was minimal. This is because there was a very high level of agreement between the artificial intelligence-based automatic speech recognition system and the experts scores. Linear regression analyses showed that the artificial intelligence-based automatic speech recognition system significantly predicted the scores of experts. According to the logistic regression analysis results, the artificial intelligence-based automatic speech recognition system was at least 93% as successful as human raters in classifying readers as poor and good. These results give us hope that reading assessments at classroom, school, regional, national, and even international levels can be conducted more accurately and economically by using artificial intelligence-based systems in the coming years.