Is ChatGPT the Next Dr. Google?
Evaluating AI Accuracy in Juvenile Idiopathic Arthritis Queries
June 19, 2025 (Vancouver) – New Arthritis Research Canada study reveals artificial intelligence can provide accurate medical information on juvenile idiopathic arthritis – the most common childhood joint disease, affecting an estimated three million kids and youth globally – regardless of geographic location.
Researchers in Canada, India, Italy, Kenya and Türkiye met online on August 29, 2024 and simultaneously asked ChatGPT the same questions related to juvenile idiopathic arthritis. They posed the questions continuously in English and without leaving the interactive chat.
The AI chatbot’s responses were 52-84% aligned with the American College of Rheumatology’s guidelines for this type of arthritis. No differences were observed across countries within responses. However, most information provided by ChatGPT was difficult to read, and word choice varied with geographical focus.
“People are using artificial intelligence tools to diagnose and manage different types of arthritis,” said Dr. Deniz Bayraktar, an Affiliate Scientist at Arthritis Research Canada, who was a co-author of this study. Another co-author, Dr. Linda Li, a Senior Scientist at Arthritis Research Canada added, “However, we need to continue to conduct research in this area to avoid potentially misleading or harmful information.”
This study suggests that large language models like ChatGPT – which can process and generate text – are promising, new applications in the healthcare field.
Prior to this research, little was known about whether asking AI tools the same questions from different countries or regions would result in the same outputs, with similar readability, text organization, and accuracy. Cultural and local factors may influence responses. However, this research shows that overall accuracy remains high when using tools like ChatGPT in different countries.
To read the full research paper, click here.
