New types of evidence

February 10, 2013 — 4 Comments

It can be difficult to figure out how to use the results of research studies (randomized controlled trials or RCTs) to make a healthcare decision. There are many other problems with RCTs that may be less obvious.

First, to perform an RCT can take years – you need to get approval from the hospital where you are performing the study because you are doing research on humans. Then you need to get funding for the study so you may need to apply for some grants. After the study is completed, the results need to be analyzed; and then a paper needs to be written and submitted for publication to a journal. It could take years from the time the results are known until the time they are published.

But just because a study is published doesn’t necessarily mean it is reliable. Sometimes researchers get funding from pharmaceutical companies which could influence the results (this is one possible type of bias in research studies). Also the way the study is performed is important. For example if there are too few people in the study, the results may not be valid.

Also, not everything important is published. Positive results are more likely to get published than negative results so this is another possible type of bias in the medical literature (Ben Goldacre wrote a recent piece in the NY Times about this).

Then it can take 17 years or more for treatments that have been proven to work to be adopted in medical practice. There are lots of reasons for this but the main one is that change is often difficult.

Given all the issues, you might think it is not worth it to perform RCTs. But the problem with just doing what seems logical is that often RCTs prove that the “logical” thing doesn’t work. We used to remove tonsils in children who had lots of sore throats, gave antibiotics to children with ear infections, gave hormones to women after menopause and put patients with a heart attack at bedrest for 14 days. We now know that these things do more harm than good.

RCTs are still the “gold standard” way to study how well treatments work and we need to make them more useful (by shortening the time to publication, including negative results in publications, getting patients to help design the studies, etc.). But we need to find innovative ways to get information to help patients make decisions when there is no RCT evidence. In short, we need to find new ways of collecting reliable evidence.

Here are a few innovative ways to collect data that are in use today:

  • Patients using sensors and smartphone apps to track what is happening in their bodies (ex: Quantified Self)
  • Patients sharing data and experiences in online discussion groups (ex: Patients Like Me)
  • Creating electronic health records that are controlled by patients to help with communication between patients and the healthcare team (ex: Patients Know Best in the UK)
  • Performing “N of 1” experiments designed by patients and their families to study what works best for them (ex: Collaborative Chronic Care Network (C3N))
  • Collecting data from patients about their experiences with treatments in a standard way (ex: DIPEx and in the UK)
  • Pooling all the data from clinical studies so that researchers can search for answers to new questions (ex: Human Studies Database)

We need to do a better job of getting the results of research studies into the hands of doctors and patients in a more timely and reliable way. But we also need to think differently about evidence and find innovative ways to learn about what works and what doesn’t work.

4 responses to New types of evidence

  1. Trevor Powell May 8, 2013 at 3:40 am

    Right on the money, in a commendably succinct review of the present situation and with a very useful conspectus of new ways to garner valuable information. This must be the way of the future in face of the inadequacies of RCTs as the sole route for progress.

    In many domains, such as late-stage cancers, patients die while RCTs drag their way through to, often unavailing, outcomes. So much more can be achieved if the shortcomings of clinical trials are fully recognised and they cease to be regarded with excessive deference as the only way to ‘true knowledge’.

Trackbacks and Pingbacks:

  1. The wisdom of patients - Care Triad - September 19, 2013

    […] only will patients be helping each other, but they will also be generating a new type of evidence that can help improve the quality of […]

  2. Personal experiments | Care Triad - May 28, 2015

    […] there are lots of problems with research studies – they take a long time to complete, the patients in the studies are very carefully selected […]

  3. Clinical trial problems in the news (again) | Care Triad - September 30, 2015

    […] Randomized controlled trials (RCTs) may have lots of problems, but they remain the “gold standard” to determine whether a drug or treatment works. A recent study published in the BMJ, again raises concerns about trusting clinical trial results. The situation is outlined in an article in the New York Times about the antidepressant drug Paxil (paroxetine) and its safety in teens. The original study was published in 2001 in the Journal of the American Academy of Child and Adolescent Psychiatry (JAACAP) and the authors concluded that Paxil is “generally well tolerated and effective” for adolescents with major depression. However, since that time, experts have questioned whether the data really supports this conclusion and whether Paxil is really safe in young adults with depression. […]

Leave a Reply

Text formatting is available via select HTML. <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>