December 20th, 2007 by Dr. Val Jones in Book Reviews
No Comments »
Which news source do you trust more: The New York Times or The National Enquirer? Which news reporter would you trust more: Charlie Gibson or Jerry Springer? As it turns out, medical journals and science researchers run the gamut from highly credible and respected to dishonest and untrustworthy. So as we continue down this road of learning how to evaluate health news, let’s now turn our attention to pillar number one of trustworthy science: credibility.
In medical research, I like to think of credibility in three categories:
1. The credibility of the researcher: does the researcher have a track record of excellence in research methodology? Is he or she well-trained and/or have access to mentors who can shepherd along the research project and avoid the pitfalls of false positive “artifacts?” Has the researcher published previously in highly respected, peer reviewed journals?
2. The credibility of the research: does the study design reflect a clear understanding of potential result confounders and does it control for false positive influences, especially the placebo effect?
3. The credibility of the journal that publishes the research: top tier journals have demonstrated a track record of careful peer review. They have editorial boards of experts who are trained in research methodology and are screened for potential conflicts of interest that could inhibit an objective analysis of the research that they review. The importance of careful peer review must not be underestimated. Some say that the quality of a product is only as good as its quality control system. Top tier journals have the best quality control systems, and the articles they publish must undergo very careful scrutiny before they are published.
So as a lay person, how do you evaluate the credibility of a health news report? In practical terms, here’s what I’d recommend:
1. Look at the name of the journal reference – where was the research published? Is it from a top tier journal? R. Barker Bausell considers the following journals to be “top tier:” The New England Journal of Medicine (NEJM), The Journal of the American Medical Association (JAMA), Annals of Internal Medicine, Nature, and Science. I might cast a slightly larger net, but no one will argue that these are certainly some of the most respected journals in medicine and science.
2. Look at the study design described in the research article abstract. Was it a randomized, controlled, double-blind, placebo-controlled trial? Were there more than 50 subjects in each group? Did the authors overstate their conclusions? This sort of analysis is challenging to the lay person – so do it if you can, but if it proves too difficult, fall back on credibility check #1.
3. Look at the primary author of the research. Search for his/her name in the National Library of Medicine’s Medline database and see what other research he or she has done, and where it was published.
If the news report is based on credible research, you may feel confident in taking the results more seriously (so long as the media is representing them accurately). But before you hang your hat on a journal’s reputation, let’s take a look at the other 2 pillars of trustworthy science: plausibility and reproducibility. These two will help you navigate your way through the vast gray zone, where the credibility check doesn’t pass with flying colors – or maybe you’re dealing with neither Charlie Gibson nor Jerry Springer.This post originally appeared on Dr. Val’s blog at RevolutionHealth.com.
December 19th, 2007 by Dr. Val Jones in Book Reviews
No Comments »
Have you ever been surprised and confused by what seem to be conflicting results from scientific research? Have you ever secretly wondered if the medical profession is comprised of neurotic individuals who change their mind more frequently than you change your clothes? Well, I can understand why you’d feel that way because the public is constantly barraged with mixed health messages. But why is this happening?
The answer is complex, and I’d like to take a closer look at a few of the reasons in a series of blog posts. First, the human body is so incredibly complicated that we are constantly learning new things about it – how medicines, foods, and the environment impact it from the chemical to cellular to organ system level. There will always be new information, some of which may contradict previous thinking, and some that furthers it or ads a new facet to what we have already learned. Because human behavior is also so intricate, it’s far more difficult to prove a clear cause and effect relationship with certain treatments and interventions, due to the power of the human mind to perceive benefit when there is none (placebo effect).
Second, the media, by its very nature, seeks to present data with less ambiguity than is warranted. R. Barker Bausell, PhD, explains this tendency:
1. Superficiality is easier to present than depth.
2. The media cannot deal with ambiguity, subtlety, and diversity (which always characterizes scientific endeavors involving new areas of investigation or human behavior in general)
3. The bizarre always gets more attention than the usual.
I really don’t blame the media – they’re under intense pressure to find interesting sound bites to keep peoples’ attention. It’s not their job to present a careful and detailed analysis of the health news that they report. So it’s no wonder that a research paper suggesting that a certain herb may influence cancer cell protein expression in a Petri dish becomes: herb is new cure for cancer! Of course, many media outlets are more responsible in their reporting than that, but you get the picture.
And thirdly, the scientific method (if not carefully followed in rigorous, randomized, placebo-controlled trials) is a set up for false positive tests. What does that mean? It means that the default for your average research study (before it even begins) is that there will be a positive association between intervention and outcome. So I could do a trial on, say, the potential therapeutic use of candy bars for the treatment of eczema, and it’s likely (if I’m not a careful scientist) that the outcome will show a positive correlation between the two.
There are many reasons for false positive results (e.g. wrongly ascribing effectiveness to a given therapy) in scientific research. “Experimental artifacts” as they’re called, are very common and must be accounted for in a study’s design. For fun let’s think about how the following factors stack the deck in favor of positive research findings (regardless of the treatment being analyzed):
1. Natural History: most medical conditions have fluctuating symptoms and many improve on their own over time. Therefore, for many conditions, one would expect improvement during the course of study, regardless of treatment.
2. Regression to the Mean: people are more likely to join a research study when their illness/problem is at its worst during its natural history. Therefore, it is more likely that the symptoms will improve during the study than if they joined at times when symptoms were not as troublesome. Therefore, in any given study – there is a tendency for participants in particular to improve after joining.
3. The Hawthorne Effect: people behave differently and experience treatment differently when they’re being studied. So for example, if people know they’re being observed regarding their work productivity, they’re likely to work harder during the research study. The enhanced results therefore, do not reflect typical behavior.
4. Limitations of Memory: studies have shown that people ascribe greater improvement of symptoms in retrospect. Research that relies on patient recall is in danger of increased false positive rates.
5. Experimenter Bias: it is difficult for researchers to treat all study subjects in an identical manner if they know which patient is receiving an experimental treatment versus a placebo. Their gestures and the way that they question the subjects may set up expectations of benefit. Also, scientists are eager to demonstrate positive results for publication purposes.
6. Experimental Attrition: people generally join research studies because they expect that they may benefit from the treatment they receive. If they suspect that they are in the placebo group, they are more likely to drop out of the study. This can influence the study results so that the sicker patients who are not finding benefit with the placebo drop out, leaving the milder cases to try to tease out their response to the intervention.
7. The Placebo Effect: I saved the most important artifact for last. The natural tendency for study subjects is to perceive that a treatment is effective. Previous research has shown that about 33% of study subjects will report that the placebo has a positive therapeutic effect of some sort.
So my dear readers – if the media wants to get your attention with exaggerated representations of research findings, and the research findings themselves are stacked in favor of reporting an effect that isn’t real… then how on earth are we to know what to make of health news? Luckily, R. Barker Bausell has explained all of this really well in his book and I will attempt to summarize the following principles in the next few posts:
1. The importance of credible scientific evidence
2. The importance of plausible scientific evidence
3. The importance of reproducible scientific evidenceThis post originally appeared on Dr. Val’s blog at RevolutionHealth.com.
December 17th, 2007 by Dr. Val Jones in Uncategorized
No Comments »
It’s time for the 4th annual Medical Blogger awards… nominate your favorites at MedGadget:
This competition
is designed to recognize the very best from the medical blogosphere,
and to highlight the diversity and excitement of the world of medical
blogs.
The categories for this year’s awards will be:
— Best Medical Weblog
— Best New Medical Weblog (established in 2007)
— Best Literary Medical Weblog
— Best Clinical Sciences Weblog
— Best Health Policies/Ethics Weblog
— Best Medical Technologies/Informatics Weblog
— Best Patient’s Blog
This post originally appeared on Dr. Val’s blog at RevolutionHealth.com.
December 12th, 2007 by Dr. Val Jones in True Stories
1 Comment »
A few months ago I wrote about my dear friend who was diagnosed with stage IV colon cancer just after having her first baby. Her optimism and positive focus was inspirational to me, and I marveled at her ability to live life unencumbered by prognosis.
Today I am happy to report that she has completed her 7th round of chemotherapy (with avastin, oxaliplatin, 5FU, and leukovorin) and her liver tumors have already shrunk by two-thirds. She is bothered by neuropathy (burning sensations in her hands and feet – a known side effect of this drug regimen) and has lost the ability to taste food fully, but otherwise she is maintaining her weight and her blood counts are good. She told me that her plan is to complete 12 rounds of chemo and then meet with a liver surgeon to consider surgical removal of the remaining tumors. Miraculously, her doctors believe that she may be a candidate for liver resection and eventual cure.
This couldn’t be any better news and I am so pleased to be able to celebrate my friend’s success. I have learned a great deal myself about the journey, and about how many people are willing to reach out and help those who are struggling. As for my friend, she has greatly benefited from:
1. CarePages – an online gathering place where friends and family can find out the latest news about her progress, and leave well wishes and virtual gifts. CarePages is part of the Revolution Health family.
2. The Colon Cancer Alliance (CCA) – this wonderful non-profit organization linked my friend up with a mentor who has been through a similar treatment regimen and diagnosis and can speak to her about what to expect. The CEO of CCA has even taken time out of his busy schedule to make sure that my friend gets the best support available and has put her in touch with top liver surgeons.
3. Dr. Lenz’s Colon Cancer Blog – Dr. Lenz is a leading medical oncologist and Co-director of both the Colorectal Center and the Gastrointestinal (GI) Oncology Program at USC/Norris Comprehensive Cancer Center in Los Angeles. He also prepares 3 fresh blog posts per week here at Revolution Health, to make sure that cancer patients have access to the latest research and information about colon cancer.
With incredible advances in evidence-based cancer therapies and these terrific online resources, colon cancer patients have a brighter future than ever before. I’m so pleased that my friend is doing well, and I’m grateful for the many people and organizations that have touched her life. Expect another update on her progress in March!This post originally appeared on Dr. Val’s blog at RevolutionHealth.com.
December 11th, 2007 by Dr. Val Jones in News
No Comments »
I was troubled by an article that I read in the Washington Post today. Apparently a common cold virus (adenovirus) has mutated unexpectedly into a killer strain (with a death rate as high as 23% for those infected in Oregon). The CDC has been carefully tracking this virus over the past year, and has noted outbreaks in Oregon, Washington State, Texas, and New York. Genetic studies of this virus suggest that it is similar to adenovirus 14, first described in Holland in 1955, but has very recently mutated into a more virulent form.
The interesting (and frightening) thing about this virus is that it’s so strong that it can kill young, otherwise healthy people, rather than the usual “at risk” populations of the very young and the very old. There’s no way to know if this virus will spark an epidemic, or whether it will die down and disappear for now.
Those who have had the virus, and lived to tell about it, say that it came on like a usual cold but then instead of getting better at the usual 4-7 day point, their symptoms got worse, and high fevers, cough, and difficulty breathing ensued. Some folks have even required mechanical ventilation in the ICU setting to help them breathe.
This virus reminds me that we are very vulnerable to unexpected attacks from infectious diseases. All the while we docs had our eyes on bird flu and SARS, this little cold virus mutated into a killer. If you have a high fever with difficulty breathing, please see your doctor right away. Let’s hope this epidemic threat dissipates before it can gain a real foothold.This post originally appeared on Dr. Val’s blog at RevolutionHealth.com.
