Good research design is critical to the advancement of science, but what exactly constitutes good research design? How should patients evaluate research study results, and why is this so important?

 

To get answers, Sharan Rai, a Consumer Advisory Board member, interviewed Charles Goldsmith, Ph.D., a clinical epidemiologist and biostatistician with the Arthritis Research Centre of Canada. Dr. Goldsmith is the Milan and Maureen Ilich/Merck Chair in Statistics for Arthritis and Musculoskeletal Diseases and a Professor in the Faculty of Health Sciences at Simon Fraser University.

 

Dr. Goldsmith has been involved in a variety of studies related to musculoskeletal and surgical conditions for many years, and currently contributes to the quality of research conducted at the Arthritis Research Centre of Canada.  Here is what he had to say:
 
 

Dr. Goldsmith, what is “biostatistics” and how did you first become interested in the field?

 
Biostatistics is the application of statistics to a variety of topics in the medical and biological sciences. I first became interested in the field of statistics during my third year of undergraduate study at the University of Manitoba. I found it easy to understand and really enjoyed it.

 

After obtaining a masters degree in statistics I completed my Ph.D. in the field at North Carolina State University.  At that time I was interested in experimental design and mathematical statistics.
 
 

As a biostatistician, what is your role in research study development at ARC?

 

Having a biostatistician is an invaluable resource to creating a good research design.

 

I play a role in all phases of research. Once the investigators decide on the research they want to do, they come to me. Considerations include:  What kind of study are they proposing? Are they looking at whether a drug or therapy works?  What comparisons are they making? For example, if it is a drug trial, we must consider if they are going to compare its effectiveness against a placebo (a drug containing no active ingredient) or another drug.

 

In addition to my role in the planning of a study, I assist with the data management. Are the data properly collected and recorded without mistakes? What will happen to the data at the end of the study?  What steps will be taken to ensure that the patients’ privacy is well protected?

 

Together the statistician and researcher develop a research plan that makes sense and is feasible. Statisticians also help to plan how the investigators will best assess the data to determine whether or not a particular therapy works.
 
 

Tell us about some of the research projects you are involved with at ARC.

 

Dr. Diane Lacaille and I, with others, are in the beginning stages of a five-year randomized trial to investigate the use of a web based, self management, “return to work” program for patients with rheumatoid arthritis (RA). The onset of rheumatoid arthritis is usually in middle age, but it often occurs in the twenties or thirties, meaning that they still have a significant working life ahead of them. Unfortunately, RA sometimes makes it difficult for patients to do certain work-related tasks, and I think some employers have not been able to accommodate these patients’ needs.  The study will measure the level of success in returning to work for program participants versus a control group who do not receive the return to work program. The study is funded by the Canadian Institute for Health Research (CIHR).

 

We see that a diagnosis of RA often means that people become unemployed, and we think that is a disaster for patients, as well as a great loss for Canadian society. We are hoping we can change that. Once the results from this study are available, the researchers will determine if there is benefit to participating in the program. If this proves to be the case, it will be implemented in a larger community. However, if little or no benefit is shown, then resources can be directed elsewhere.  This research will help us to invest wisely in helping people with arthritis to continue to participate in the workplace.
 
 

Good research has as little bias as possible. Are there biases that cannot be prevented in medical research?

 

A big issue when considering good research design is the presence of biases.

 

One of the biggest biases is believing that you can properly evaluate a drug or therapy without a randomized trial. The people who make choices about who is going to get the therapy can often slant the results of the therapy. 

 

If you were our measurement person and you know that a particular patient was in the drug group and another patient was in the non-drug group, you might alter the number that you record for the measurement in order to make the drug look good. The result would be influenced by you and not the drug’s performance.
 
 

So how are biases avoided when designing an experiment?

 

One of the things that we try to do for everyone involved in the study is to use what is called “blinding” to ensure the researchers cannot determine what group the patients are in.

 

There are lots of biases that can creep in. If you think about it, when people are involved in a study that takes a year to carry out, they will likely be doing lots of things other than taking the therapy that the research project is investigating.  Those additional activities could affect results.

 

Currently, randomization is the procedure that best deals with potential biases, such as that used in a randomized controlled trial or RCT. Randomization does not avoid biases all of the time, but it is very helpful in minimizing the impact. We try to design the study in such a way as to avoid biases but it’s not always possible to do that.
 
 

The patients/consumers are exposed to a lot of research in the media and on the Internet. How should the public look at research study results when trying to assess their relevancy?

 

The evolution of technology has made all sorts of information readily accessible by patients. When comparing two treatments, the first thing that you should determine is whether or not the study was a randomized trial. In studies about treatments, this is a signal that the study was not done as well as it should have been. And, if the results are based on a single personal story, then it is the worst kind of evidence.
 
 

Patients often ask, “Do these research results apply to me? How would you answer that question?

 

There are a number of factors to consider when ascertaining whether a particular study could be relevant (and even beneficial) to you.

 

First, you will want to know something about the study participants.  Generally speaking, in every published paper there is something called Table 1 which is a description of the patients that were in the study. Table 1 will contain information such as their diagnosis, their age, and their gender. If you read over the information in this table and you see yourself (that is to say you match the criteria), then that would be your first indicator that the results could apply to you.  However, even if you do not directly identify with the study participants, talk to your doctor. Your doctor may decide that the results could apply to you as well.

 

Next, you will want to consider whether the treatment is well explained. Additionally, you should understand what was being measured [fatigue, pain, what’s happening to the cartilage, etc.]. Consideration should also be given to the duration and location of the study; it is important to ask how long the study lasted. If you’re expecting to take a therapy and use it for the rest of your life, and the study only investigated it for three months, then that would not be very convincing. To increase your confidence in the study, you would want to know if the results have been repeated in other studies.

 

Sometimes the best research available is not a randomized study.  Studies may look at large government databases. Analyzing these databases can identify trends, concerns and areas for further study. [ARC has completed many of these studies.]

 

Finally, always look at the type of publication. Ads on TV do not rate the quality of the research. And websites have almost no quality control, so you have to be wary of who produced them.
 
 

Does it make a difference having patient involvement in the research process?

 

While patients, especially those who have been recently diagnosed, may feel powerless and entirely at the hands of their doctors, there is a way to make a difference. Getting involved in the research process can make an enormous impact on the treatment of arthritis. There is an opportunity to make our research more relevant, right from the design phase. Beyond participating in the research study, patients can also help to explain research results to the public.

 

The reason I like having the Consumer Advisory Board members around to consult and work with is really this — they can tell us what it is like to be on the other end of all the things we are researching, and they can advise us whether they think what we are doing may be helpful to them and to others living with arthritis.  In short, they help to keep our compasses pointing in the right direction.

 

CAB members value having Dr. Goldsmith on ARC’s research team and would like to thank him for helping us understand more about the research process and how to attempt to identify credible research.

 

 

Randomization refers to the random allocation of study participants to receive one or the other treatments under investigation.

Caution:  Teams of researchers and experts review published health research and use a structured process to determine how confident they are in adopting the results.  We encourage our subscribers to discuss any research results with their health care providers.