Assessing the reliability and usefulness of ChatGPT responses on intermittent catheterization queries: A critical analysis


Aras N., ÇALIŞKAN N.

International Journal of Urological Nursing, cilt.18, sa.3, 2024 (ESCI) identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 18 Sayı: 3
  • Basım Tarihi: 2024
  • Doi Numarası: 10.1111/ijun.12428
  • Dergi Adı: International Journal of Urological Nursing
  • Derginin Tarandığı İndeksler: Emerging Sources Citation Index (ESCI), Scopus, CINAHL
  • Anahtar Kelimeler: ChatGPT, health information, intermittent catheterization, neurogenic bladder
  • Gazi Üniversitesi Adresli: Evet

Özet

The rapid development of technology has caused patients to start using artificial intelligence based applications more frequently to learn about their diseases. The aim of this study was to evaluate the reliability and usefulness of responses to questions related to intermittent catheterization using the large language model defined as ChatGPT. Ten questions were created based on the most searched keywords on Google Trends, the most searched questions on ChatGPT and the clinical experiences of the authors. The created questions were asked to ChatGPT and the answers were evaluated by five raters with seven-point Likert-type reliability and usefulness scales. The highest score in terms of reliability and usefulness was the response to “Is there a difference between the catheters used for intermittent catheterization in men and women?”(mean 6.4 ± 0.8; 6.0 ± 1.0). The lowest average reliability and usability score was for the response to “What is intermittent catheterization and how is it performed?”(mean 2.8 ± 0.8; 2.8 ± 0.8). ChatGPT is relatively very reliable and moderately useful in providing patients with information about intermittent catheterization, but it may give incorrect and misleading answers. It should be remembered that simple errors in intermittent catheterization can lead to major problems.