A Rasch Measurement Approach to Compare Instructor and ChatGPT Scoring in Higher Education


Çapa Aydın Y., Arslan B. Z., Özcan B.

21st European Association for Research on Learning and Instruction (EARLI) Conference , Graz, Austria, 25 - 29 August 2025, pp.1-2, (Summary Text)

  • Publication Type: Conference Paper / Summary Text
  • City: Graz
  • Country: Austria
  • Page Numbers: pp.1-2
  • Gazi University Affiliated: Yes

Abstract

It is significant to investigate what GenAI can offer for personalized learning and instruction, particularly concerning the assessment of learning in higher education. As GenAI tools can be used to score student tasks, it is worth investigating the credibility of AI-generated scores. The current study aims to compare teacher assessment to AI-generated assessment. For this purpose, 38 assignments of undergraduate students were scored by one of the most used AI tools, ChatGPT 4.0, and scored by one instructor, using the same analytical rubric. Findings indicated that the assignments received lower but consistent scores from the ChatGPT than the instructor. Further research should investigate the potential biases of AI compared to instructor assessments.