Better Health: Smart Health Commentary Better Health (TM): smart health commentary

Article Comments

The Three Pillars Of Trustworthy Science: Credibility, Plausibility, and Reproducibility

Which news source do you trust more: The New York Times or The National Enquirer? Which news reporter would you trust more: Charlie Gibson or Jerry Springer? As it turns out, medical journals and science researchers run the gamut from highly credible and respected to dishonest and untrustworthy. So as we continue down this road of learning how to evaluate health news, let’s now turn our attention to pillar number one of trustworthy science: credibility.

In medical research, I like to think of credibility in three categories:

1. The credibility of the researcher: does the researcher have a track record of excellence in research methodology? Is he or she well-trained and/or have access to mentors who can shepherd along the research project and avoid the pitfalls of false positive “artifacts?” Has the researcher published previously in highly respected, peer reviewed journals?

2. The credibility of the research: does the study design reflect a clear understanding of potential result confounders and does it control for false positive influences, especially the placebo effect?

3. The credibility of the journal that publishes the research: top tier journals have demonstrated a track record of careful peer review. They have editorial boards of experts who are trained in research methodology and are screened for potential conflicts of interest that could inhibit an objective analysis of the research that they review. The importance of careful peer review must not be underestimated. Some say that the quality of a product is only as good as its quality control system. Top tier journals have the best quality control systems, and the articles they publish must undergo very careful scrutiny before they are published.

So as a lay person, how do you evaluate the credibility of a health news report? In practical terms, here’s what I’d recommend:

1.  Look at the name of the journal reference – where was the research published? Is it from a top tier journal? R. Barker Bausell considers the following journals to be “top tier:” The New England Journal of Medicine (NEJM), The Journal of the American Medical Association (JAMA), Annals of Internal Medicine, Nature, and Science. I might cast a slightly larger net, but no one will argue that these are certainly some of the most respected journals in medicine and science.

2. Look at the study design described in the research article abstract. Was it a randomized, controlled, double-blind, placebo-controlled trial? Were there more than 50 subjects in each group? Did the authors overstate their conclusions? This sort of analysis is challenging to the lay person – so do it if you can, but if it proves too difficult, fall back on credibility check #1.

3. Look at the primary author of the research. Search for his/her name in the National Library of Medicine’s Medline database and see what other research he or she has done, and where it was published.

If the news report is based on credible research, you may feel confident in taking the results more seriously (so long as the media is representing them accurately). But before you hang your hat on a journal’s reputation, let’s take a look at the other 2 pillars of trustworthy science: plausibility and reproducibility. These two will help you navigate your way through the vast gray zone, where the credibility check doesn’t pass with flying colors – or maybe you’re dealing with neither Charlie Gibson nor Jerry Springer.This post originally appeared on Dr. Val’s blog at

You may also like these posts

Read comments »

Comments are closed.

Return to article »

Latest Interviews

IDEA Labs: Medical Students Take The Lead In Healthcare Innovation

It’s no secret that doctors are disappointed with the way that the U.S. healthcare system is evolving. Most feel helpless about improving their work conditions or solving technical problems in patient care. Fortunately one young medical student was undeterred by the mountain of disappointment carried by his senior clinician mentors…

Read more »

How To Be A Successful Patient: Young Doctors Offer Some Advice

I am proud to be a part of the American Resident Project an initiative that promotes the writing of medical students residents and new physicians as they explore ideas for transforming American health care delivery. I recently had the opportunity to interview three of the writing fellows about how to…

Read more »

See all interviews »

Latest Cartoon

See all cartoons »

Latest Book Reviews

Book Review: Is Empathy Learned By Faking It Till It’s Real?

I m often asked to do book reviews on my blog and I rarely agree to them. This is because it takes me a long time to read a book and then if I don t enjoy it I figure the author would rather me remain silent than publish my…

Read more »

The Spirit Of The Place: Samuel Shem’s New Book May Depress You

When I was in medical school I read Samuel Shem s House Of God as a right of passage. At the time I found it to be a cynical yet eerily accurate portrayal of the underbelly of academic medicine. I gained comfort from its gallows humor and it made me…

Read more »

Eat To Save Your Life: Another Half-True Diet Book

I am hesitant to review diet books because they are so often a tangled mess of fact and fiction. Teasing out their truth from falsehood is about as exhausting as delousing a long-haired elementary school student. However after being approached by the authors’ PR agency with the promise of a…

Read more »

See all book reviews »