Here we go again. Headlines across America blaring lines like, “Coffee may reduce stroke risk.”
It was a big study, but an observational study. Not a trial. Not an experiment. And, as we say so many times on this website that you could almost join along with the chorus, observational studies have inherent limitations that should always be mentioned in stories. They can’t prove cause and effect. They can show a strong statistical association, but they can’t prove cause and effect. So you can’t prove benefit or risk reduction. And stories should say that.
“The problem with this type of study is that there are too many factors unaccounted for and association does not prove causality, said Dr. Larry B. Goldstein, director of the Duke Stroke Center at Duke University Medical Center.
“Subjects were asked about their past coffee consumption in a questionnaire and then followed over time. There is no way to know if they changed their behavior,” Goldstein said.
And, he noted, there was no control for medication use or other potential but unmeasured factors.
“The study is restricted to a Scandinavian population, and it is not clear, even if there is a relationship, that it would be present in more diverse populations. I think that it can be concluded, at least in this population, that there was not an increased risk of stroke among coffee drinkers,” he said.”
When you don’t explain the limitations of observational studies — and/or when you imply that cause and effect has been established — you lose credibility with some readers. And you should. Read more »
We’re delighted to see that USA Today, Reuters, and WebMD were among the news organizations that included what an editorial writer said about an observational study linking ibuprofen use with fewer cases of Parkinson’s disease. All three news organizations used some version of what editorial writer Dr. James Bower of the Mayo Clinic wrote or said:
“Whenever in epidemiology you find an association, that does not mean causation.”
“An association does not prove causation.”
“There could be other explanations for the ibuprofen-Parkinson’s connection.”
Kudos to those news organizations. And some praise goes to the journal Neurology for publishing Dr. Bower’s editorial to accompany the study. His piece is entitled, “Is the answer for Parkinson disease already in the medicine cabinet? Unfortunately not.”
And unfortunately not all news organizations got that message. Because many don’t read the journals, so they certainly never get to the editorials. Instead, they rewrite quick hits off a wire service story. As a result, we end up with some of the following:
A FoxNews.com story was particularly deaf to Bower’s caveat, stating: “That bottle of ibuprofen in your medicine cabinet is more powerful than you may think.”
A CBSNews.com story never addressed the observational study limitation, instead whimsically writing: “Pop a pill to prevent Parkinson’s disease? A new study says it’s possible, and the pill in question isn’t some experimental marvel that’s still years away from drugstore shelves. It’s plain old ibuprofen.”Read more »
The BMJ’s statement this week that the 1998 article by Andrew Wakefield and 12 others “linking MMR vaccine and autism was fraudulent” demonstrates what a difference one journalist can make. Journalist Brian Deer played a key role in uncovering and dismantling the Wakefield story.
CNN’s Anderson Cooper had a segment worth watching, including a new interview Cooper conducted with Wakefield via Skype:
Unfortunately, journalism played a key role in promoting Wakefield’s claims. The “Respectful Insolence” blog referred to one journalist as “CBS’ resident anti-vaccine propagandist.” Around the world there were many other examples of journalists’ unquestioning acceptance of the vaccine scares.
The BMJ reminds us that “the damage to public health continues, fuelled by unbalanced media reporting and an ineffective response from government, researchers, journals, and the medical profession.”
Interesting post by the Retraction Watch blog, pointing to an interesting paper published last week in the Annals of Emergency Medicine. An excerpt from the blog post:
Over 14 years, 84 editors at the journal rated close to 15,000 reviews by about 1,500 reviewers. Highlights of their findings:
…92% of peer reviewers deteriorated during 14 years of study in the quality and usefulness of their reviews (as judged by editors at the time of decision), at rates unrelated to the length of their service (but moderately correlated with their mean quality score, with better-than average reviewers decreasing at about half the rate of those below average). Only 8% improved, and those by very small amount.
How bad did they get? The reviewers were rated on a scale of 1 to 5 in which a change of 0.5 (10%) had been earlier shown to be “clinically” important to an editor.Read more »
It’s no secret that doctors are disappointed with the way that the U.S. healthcare system is evolving. Most feel helpless about improving their work conditions or solving technical problems in patient care. Fortunately one young medical student was undeterred by the mountain of disappointment carried by his senior clinician mentors…
I am proud to be a part of the American Resident Project an initiative that promotes the writing of medical students residents and new physicians as they explore ideas for transforming American health care delivery. I recently had the opportunity to interview three of the writing fellows about how to…
I m often asked to do book reviews on my blog and I rarely agree to them. This is because it takes me a long time to read a book and then if I don t enjoy it I figure the author would rather me remain silent than publish my…
When I was in medical school I read Samuel Shem s House Of God as a right of passage. At the time I found it to be a cynical yet eerily accurate portrayal of the underbelly of academic medicine. I gained comfort from its gallows humor and it made me…
I am hesitant to review diet books because they are so often a tangled mess of fact and fiction. Teasing out their truth from falsehood is about as exhausting as delousing a long-haired elementary school student. However after being approached by the authors’ PR agency with the promise of a…
