Customer Complaint Classification with Large Language Models

Güllü M., Atagün E., Biroǧul S., BARIŞÇI N.

2nd International Conference on Artificial Intelligence, Computer, Data Sciences, and Applications, ACDSA 2025, Antalya, Türkiye, 7 - 09 Ağustos 2025, (Tam Metin Bildiri) identifier

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Doi Numarası: 10.1109/acdsa65407.2025.11166338
  • Basıldığı Şehir: Antalya
  • Basıldığı Ülke: Türkiye
  • Anahtar Kelimeler: Bert, Customer complaint, GPT, Llama, LLM, RoBERTa
  • Gazi Üniversitesi Adresli: Evet

Özet

Automatic classification of customer complaints is of great importance for companies to detect customer problems early, optimize service processes and increase customer satisfaction. Customer complaint data provided as textual data consists of very insightful information in terms of addressing the problems encountered. In this study, deep learning-based approaches (LSTM, CNN), transformer-based models (BERT, RoBERTa) and large language models (GPT-2, Llama3.1-8B) are compared using two customer complaint datasets with different class distributions. The results show that LLMs achieve higher accuracy and F1 scores in both training and test phases. In particular, the fine-tuning process applied to the large language models enabled them to be tailored to the problem and contributed to the higher performance of LLMs compared to other models.