ChatGPT shows human-level evaluation of the brain

As AI advances, its uses and capabilities in real-world applications continue to reach new heights that may surpass even human expertise. In the field of radiology, where accurate diagnosis is vital to ensure appropriate patient care, large language models like ChatGPT can improve accuracy or at least offer a good second opinion.

To test its potential, graduate student Yasuhito Mitsuyama and Associate Professor Daiju Ueda’s team at Osaka Metropolitan University School of Medicine led researchers to compare the diagnostic performance of GPT-4-based ChatGPT and radiologists on 150 preoperative brain tumor MRI reports. Based on these daily clinical notes written in Japanese, ChatGPT, two board-certified neuroradiologists, and three general radiologists were asked to provide a differential diagnosis and final diagnosis.

Their accuracy was then calculated based on the actual diagnosis after tumor removal. Results averaged 73% for ChatGPT, 72% average for neuroradiologists, and 68% average for general radiologists. Additionally, the final diagnostic accuracy of ChatGPT varied depending on whether the clinical report was written by a neuroradiologist or a general radiologist. The accuracy of neuroradiologist reports was 80%, while the accuracy of general radiologist reports was 60%.

“These results suggest that ChatGPT may be useful for preoperative MRI diagnosis of brain tumors,” said graduate student Mitsuyama. “In the future, we plan to examine big language models in other areas of diagnostic imaging to use AI to reduce the burden on physicians, increase diagnostic accuracy, and support educational environments.”

The findings were published at: European Radiology.

###

About OMU

Founded in Osaka as one of Japan’s largest public universities, Osaka Metropolitan University is committed to shaping the future of society through the “Convergence of Knowledge” and the promotion of world-class research. For more research news, visit: https://www.omu.ac.jp/en/ and follow us on social media: X, Facebook, instagram, LinkedIn.