ChatRheum: Developing a chatbot for answering rheumatology-related questions
Scientific Study Title:
ChatRheum: Developing and evaluating a source-verified large language model to answer rheumatology patient questions
Study Start Date:
2025
End Date:
2029
Why Do This Research?
Many people in Canada can’t receive timely care from a rheumatologist because there aren’t enough specialists to meet the growing demand. This research project aims to help this problem by developing a chatbot, called ChatRheum, that can accurately answer patients’ questions about rheumatology.
This research is important because if the chatbot can provide patients with accurate, easy-to-understand, and compassionate answers, it could reduce the time doctors spend answering basic questions. This will free up time for rheumatologists to see more patients more quickly. This could shorten wait times, improve access to care, and make our health system in Canada more efficient.
What Will Be Done?
This research will include three parts:
- In part 1, the research team will look for trusted information by asking both rheumatologists and patients where they usually get reliable information about rheumatic diseases; then,
- In part 2, using the trusted information and sources identified in part 1, the team will train the chatbot, ChatRheum, to answer rheumatology-related questions.
- Finally, in part 3, the team will then test how well ChatRheum performs by comparing its answers to those from other AI chatbots (like ChatGPT) and rheumatologists. They will measure how accurate, complete, easy to understand, and compassionate the answers are.
Who Is Involved?
Rheumatology patients and rheumatologists from across Canada will be surveyed to gather information on their sources of rheumatology information, and they will also serve as evaluators of ChatRheum’s answers.
How Are Equity, Diversity and Inclusion Addressed or Taken Into Consideration?
An important part of the ChatRheum development is minimizing bias and misinformation. This process begins at step 1, where we ensure that both online and in-person surveys are made available, and continues through step 3, where we evaluate ChatRheum for instances of bias or misrepresentation.
How Will/ Have Patients Contribute to This Research?
Patients are involved in all stages of this research, from patient partners helping to develop the study plan, to being involved in determining information sources and evaluating the chatbot.
The Research Team
Principal Investigators:
Carrie Ye, MD, MPH, FRCPC, Research Scientist, Arthritis Research Canada (University of Alberta)
Co-investigators:
Mohamed Abdalla, PhD (University of Alberta)
Steven Katz, MD (University of Alberta)
Ross Mitchell, MSc, PhD (University of Alberta)
Dax Rumsey, MD, MSc (University of Alberta)
Who Funded This Research?
Canadian Institutes of Health Research