Is ChatGPT the Next Dr. Google?

Evaluating AI Accuracy in Juvenile Idiopathic Arthritis Queries

June 19, 2025 (Vancouver) – New Arthritis Research Canada study reveals artificial intelligence can provide accurate medical information on juvenile idiopathic arthritis – the most common childhood joint disease, affecting an estimated three million kids and youth globally – regardless of geographic location.

Researchers in Canada, India, Italy, Kenya and Türkiye met online on August 29, 2024 and simultaneously asked ChatGPT the same questions related to juvenile idiopathic arthritis. They posed the questions continuously in English and without leaving the interactive chat.

The AI chatbot’s responses were 52-84% aligned with the American College of Rheumatology’s guidelines for this type of arthritis. No differences were observed across countries within responses. However, most information provided by ChatGPT was difficult to read, and word choice varied with geographical focus.

“People are using artificial intelligence tools to diagnose and manage different types of arthritis,” said Dr. Deniz Bayraktar, an Affiliate Scientist at Arthritis Research Canada, who was a co-author of this study. Another co-author, Dr. Linda Li, a Senior Scientist at Arthritis Research Canada added, “However, we need to continue to conduct research in this area to avoid potentially misleading or harmful information.”

This study suggests that large language models like ChatGPT – which can process and generate text – are promising, new applications in the healthcare field.

Prior to this research, little was known about whether asking AI tools the same questions from different countries or regions would result in the same outputs, with similar readability, text organization, and accuracy. Cultural and local factors may influence responses. However, this research shows that overall accuracy remains high when using tools like ChatGPT in different countries.

To read the full research paper, click here.

– 30 –

Arthritis Research Canada is the largest clinical arthritis research institution in North America. Our mission is to transform the lives of people impacted by arthritis through patient-centred research and engagement. Arthritis Research Canada’s scientific director, Dr. Diane Lacaille, leads a team of over 100 researchers, trainees and staff whose life-changing research is creating a future where people with arthritis have the knowledge and tools to triumph over pain and disability. Arthritis Research Canada is now conducting arthritis research from coast to coast with centres in British Columbia, Alberta, Quebec, and Nova Scotia and scientists affiliated with ten major Canadian universities: University of British Columbia, Simon Fraser University, University of Calgary, University of Alberta, Western University, Université Laval, McGill University, Université de Montréal, Université de Sherbrooke and Dalhousie University. Arthritis Research Canada is leading research aimed at preventing arthritis, facilitating early diagnosis, finding new and better treatments, and improving quality of life.

For more information or to arrange an interview, please contact:

Heather Caulder
Senior Officer, Marketing and Communications
Arthritis Research Canada
604.207.4010
hcaulder@arthritisresearch.ca