ChatRheum: Developing a Chatbot for Answering Rheumatology-Related Questions
Research_ChatRheum-chat-bot_425x296

Scientific Study Title:

ChatRheum: Developing and evaluating a source-verified large language model to answer rheumatology patient questions

Start Date: 

End Date:

Carrie Ye
Research Scientist, Rheumatology, MD, MPH, FRCPC

Ready to Participate?

Life-changing arthritis research is only possible with your help!

Why do this research?

Reason For Research

Many people in Canada can’t receive timely care from a rheumatologist because there aren’t enough specialists to meet the growing demand. This research project aims to help this problem by developing a chatbot, called ChatRheum, that can accurately answer patients’ questions about rheumatology.

This research is important because if the chatbot can provide patients with accurate, easy-to-understand, and compassionate answers, it could reduce the time doctors spend answering basic questions. This will free up time for rheumatologists to see more patients more quickly. This could shorten wait times, improve access to care, and make our health system in Canada more efficient.

Methodology

Execution of Research

This research will include three parts:

  1. In part 1, the research team will look for trusted information by asking both rheumatologists and patients where they usually get reliable information about rheumatic diseases; then,
  2. In part 2, using the trusted information and sources identified in part 1, the team will train the chatbot, ChatRheum, to answer rheumatology-related questions.
  3. Finally, in part 3, the team will then test how well ChatRheum performs by comparing its answers to those from other AI chatbots (like ChatGPT) and rheumatologists. They will measure how accurate, complete, easy to understand, and compassionate the answers are.

Who is involved?

Involvement

Rheumatology patients and rheumatologists from across Canada will be surveyed to gather information on their sources of rheumatology information, and they will also serve as evaluators of ChatRheum’s answers.

How Are Equity, Diversity and Inclusion Addressed or Taken Into Consideration?

An important part of the ChatRheum development is minimizing bias and misinformation. This process begins at step 1, where we ensure that both online and in-person surveys are made available, and continues through step 3, where we evaluate ChatRheum for instances of bias or misrepresentation.

How Will/ Have Patients Contribute to This Research?

Patients are involved in all stages of this research, from patient partners helping to develop the study plan, to being involved in determining information sources and evaluating the chatbot.

Related Research