Better Health: Smart Health Commentary Better Health (TM): smart health commentary

Article Comments

Reproducibility: The Final Pilar of Trustworthy Science

This is my final post in a series inspired by Dr. Barker Bausell’s recent book, “Snake Oil Science: The Truth About Complementary and Alternative Medicine.” Since I began this series, the New York Times has published a rave review of Bausell’s book, which only further confirms the importance of Bausell’s contributions.

Although Bausell’s main thesis is that there are currently no large, randomized controlled trials (published in leading medical journals) demonstrating the effect of any CAM therapy beyond placebo, I have chosen to highlight some of his thinking about research methodology as it applies to the medical literature in general.

So far I have explained why most research (if not carefully designed) will lead to a false positive result. This inherent bias is responsible for many of the illusionary treatment benefits that we hear about so commonly through the media (whether they’re reporting about CAM or Western medicine), because it is their job to relay information in an entertaining way more so than an accurate manner (i.e. good science makes bad television).Then I explained a three step process for determining the trustworthiness of health news and research. We can remember these steps with a simple mnemonic: C-P-R.

The C stands for credibility- in other words, “consider the source” – is the research published in a top tier medical journal with a scientifically rigorous review process?

The P stands for plausibility- is the proposed finding consistent with known principles of physics, chemistry, and physiology or would accepting the result require us to suspend belief in everything we’ve learned about science to date?

And finally we arrive at R – reproducibility. If the research study were repeated, would similar results be obtained?

This third and final pillar of trustworthy science is a simple, but sometimes forgotten, principle. If there is a true cause and effect relationship observed by the researcher, then surely that cause and effect can be demonstrated again and again under the same conditions. Touching a hot stove burner always results in a burned hand. No matter how frequently you test this causal relationship, the result will be similar.

Sometimes conflicting results are obtained by repeating a study. When this happens, the reader should be careful in interpreting the conclusions – there may be a flaw in the study design, or it may be that the conclusions drawn were inaccurate. There could have been a false positive result, or no appreciable effect of the treatment under consideration, therefore leaving the results to chance. Flipping a coin gives you heads one minute and tails the next. Yet a person unfamiliar with coins could conclude (after one flip) that it has a head on both sides. In the end, therefore, one can be more confident in a study’s result if it is born out by other studies.

And so as we conclude this series, I hope that you now feel well equipped to perform CPR (credibility, plausibility, reproducibility checks) on health news. A little healthy skepticism can protect your brain from all the mixed health messages that barrage us each day. At the very least, now you’ll appreciate why most health news reports include an expert quote stating something to the effect of “it’s too early to know for sure if these findings are relevant.” That statement may be the most trustworthy of the entire report.


Next up: Shannon Brownlee’s book “Overtreated: Why Too Much Medicine Is Making Us Sicker And Poorer.” Shannon and I corresponded about this book two years ago, so I’m looking forward to seeing how it has turned out. Once I’ve finished it I’ll give you my thoughts here in this blog.This post originally appeared on Dr. Val’s blog at

You may also like these posts

Read comments »

Comments are closed.

Return to article »

Latest Interviews

IDEA Labs: Medical Students Take The Lead In Healthcare Innovation

It’s no secret that doctors are disappointed with the way that the U.S. healthcare system is evolving. Most feel helpless about improving their work conditions or solving technical problems in patient care. Fortunately one young medical student was undeterred by the mountain of disappointment carried by his senior clinician mentors…

Read more »

How To Be A Successful Patient: Young Doctors Offer Some Advice

I am proud to be a part of the American Resident Project an initiative that promotes the writing of medical students residents and new physicians as they explore ideas for transforming American health care delivery. I recently had the opportunity to interview three of the writing fellows about how to…

Read more »

See all interviews »

Latest Cartoon

See all cartoons »

Latest Book Reviews

Book Review: Is Empathy Learned By Faking It Till It’s Real?

I m often asked to do book reviews on my blog and I rarely agree to them. This is because it takes me a long time to read a book and then if I don t enjoy it I figure the author would rather me remain silent than publish my…

Read more »

The Spirit Of The Place: Samuel Shem’s New Book May Depress You

When I was in medical school I read Samuel Shem s House Of God as a right of passage. At the time I found it to be a cynical yet eerily accurate portrayal of the underbelly of academic medicine. I gained comfort from its gallows humor and it made me…

Read more »

Eat To Save Your Life: Another Half-True Diet Book

I am hesitant to review diet books because they are so often a tangled mess of fact and fiction. Teasing out their truth from falsehood is about as exhausting as delousing a long-haired elementary school student. However after being approached by the authors’ PR agency with the promise of a…

Read more »

See all book reviews »