The Friday Funny: Hospital Length Of Stay
Latest Posts
Bad Science: Nine Ways To Dress Up Flawed Research
This post is a follow up to my book review of Bad Science, located here.
***
I couldn’t help but feel unusually depressed by Dr. Ben Goldacre’s exposé of researchers who resort to trickery to get their articles published in peer-reviewed journals. There are a number of ways to manipulate data and many ways that flawed research is presented to enhance its chances of publication.
Before we get started, I need to point out that “negative trials” – or research results that don’t corroborate the investigator’s original hypothesis/es – are much less likely to be published. People (and/or publishers) are far more interested in finding a needle in a haystack, than hearing that no needle could be found. This is a driving force in all manner of mathematical convolutions aimed at demonstrating something interesting and to warrant publication. After all, who can blame the researchers for wanting to get their research published, and to have it make a meaningful contribution to their field of study? Who wants to toil for months to years on end, only to discover that their hypotheses were not born out by experimentation, and in fact no helpful conclusions may be drawn whatsoever?
And so, with this intense pressure to find something meaningful in one’s research (either for profit or personal satisfaction and professional advancement) – there are some typical stragegies that researchers use to make something out of nothing. Ben Goldacre reviews these strategies in the voice of an unscrupulous senior pharmaceutical investigator, giving advice to his junior colleague. (Parenthetically, it reminded me of The Screwtape Letters – an amusing book written by C.S. Lewis, featuring the imaginary advice of a senior demon to his junior counterpart as they tempt humans to evil.)
(This passage is taken directly from pages 192-193 of Bad Science)
1. Ignore the protocol entirely. Always assume that any correlation proves causation. Throw all your data into a spreadsheet programme and report – as significant – any relationship between anything and everything if it helps your case. If you measure enough, some things are bound to be positive by sheer luck.
2. Play with the baseline. Sometimes, when you start a trial, quite by chance the treatment group is already doing better than the placebo group. If so, then leave it like that. If, on the other hand, the placebo group is already doing better than the treatment group at the start, then adjust for the baseline in your analysis.
3. Ignore dropouts. People who drop out of trials are statistically much more likely to have done badly, and much more likely to have had side effects. They will only make your drug look bad. So ignore them, make no attempt to chase them up, do not include them in your final analysis.
4. Clean up the data. Look at your graphs. There will be some anomalous ‘outliers,’ or points which lie a long way from the others. If they are making your drug look bad, just delete them. But if they are helping your drug look good, even if they seem to be spurious results, leave them in.
5. The best of five… no… seven… no… nine! If the difference between your drug and placebo becomes significant four and a half months into a six-month trial, stop the trial immediately and start writing up the results: things might get less impressive if you carry on. Alternatively, if at six months the results are ‘nearly significant,’ extend the trial by another three months.
6. Torture the data. If your results are bad, ask the computer to go back and see if any particular subgroups behaved differently. You might find that your drug works very well in Chinese women aged fifty-two to sixty-one. ‘Torture the data and it will conhfess to anything’ as they say at Guantanamo Bay.
7. Try every button on the computer. If you’re really desperate, and analysing your data the way you planned doesn’t give you the results you wanted, just run the figures through a wide selection of other statistical tests, even if they are entirely inappropriate, at random.
8. Publish wisely. If you have a good trial, publish it in the biggest journal you can possibly manage. If you have a positive trial, but it was a completely unfair test, then put it in an obscure journal… and hope that readers are not attentive enough to read beyond the abstract to recognize its flaws.
9. Do not publish. If your finding is really embarrassing, hide it away somewhere and cite ‘data on file.’ Nobody will know the methods, and it will only be noticed if someone comes pestering you for the data to do a systematic review. Hopefully that won’t be for ages.
Where Have All the Family Practice Doctors Gone? First Aid for Primary Care
By Alan W. Dappen, MD; Steve Simmons, MD; Valerie Tinley, FNP of Doctokr Family Medicine
We are a family doctor, an internist and a family nurse practitioner working on the front line of the American health care system. We share a moral and ethical duty to protect the health of our patients along with all our colleagues who labor daily doing the same.We as Americans are proud of what has long been considered a first-rate health care system. Sadly, this system is broken despite our best efforts. Americans spend much more per capita for care as any other country. The World Health Organization has graded our care as 37th “best” in the world. Even worse, American citizens were the least satisfied with their medical care compared to the next five leading socialized industrialized countries, including England, Germany, Canada, Australia and New Zealand. There are many things wrong. Let’s examine a few:
Primary care medicine in America is gasping for its last breath. Internists, family doctors, pediatricians (whom health experts consider essential to a robust and cost-effective delivery system) are leaving primary care in droves. The number of newly trained generalist doctors has plummeted so fast that extinction of the generalist doctor has been forecasted within 20 years by both the American Academy of Family Practice and the American College of Physicians.
Patients are angry and exasperated with long delays, poor service and confusing and redundant paperwork. To date 17% of us are uninsured and this number will quickly grow in a deepening recession.
Employers face a huge cost burden as health insurance prices go through the roof. CEOs consistently say the runaway costs in health care benefits (which double in price every seven to ten years) threaten the viability of their companies. Since 2000, the number of small businesses offering health insurance has dropped 8%.
Health insurance companies are making so much money that several states have motioned legislation compelling insurance companies to disclose the percentage of premiums spent on actual medical care. Not surprisingly, their lobbyists are resisting. It is not uncommon for insurance companies to keep 30-40% of every dollar for “administration” and profits. Many of these companies are on record reaffirming their commitment to shareholders and short-term profits.
Doctokr (“doc-talker”) Family Medicine is a medical practice that was created to respond to the conflicts and problems listed above. We have worked to resuscitate the soul of the Marcus Welby-style patient-focused physician while adding technology to deliver fast, responsive and informed care. All fees are transparent and time-based and are the responsibility of our patients to pay. All parties that interfere with the doctor patient relationship or increase our costs have been removed from the equation. The practice delivers “concierge level” services: 24/7 access, connectivity to the doctor no matter where our patients are located, same day office visits for those that need to be seen, even house calls for those unable to get to our office. By removing the hurdles and restoring transparency and trust, 75% of our clients get their entire primary care needs met for $300.00 a year.
This post is written by three medical professionals who stopped waiting for someone else to find a solution and are actively changing primary care in ways that dramatically improve quality, convenience and access, while drastically reducing costs. The US deserves excellent health care and it must be done right. To understand why we would bother to “walk the walk,” we ask your indulgence and participation while we “talk the talk.” We hope this format will educate and inform you in ways that move you to participate in your care. Health care is about you, just as much as it about us, because we are all patients. We all have a stake in shaping the inevitable need for reform.
The next upcoming topics:
- Where did the Marcus Welby, MD-style of primary care go and how can we get it back?
- How have you as a patient lost control of your body and health?
- Turning the primary care model upside down: What does primary care need to do to reinvent itself so that it serves its patients without other conflicting interests?
- Begin the exploration of the unexamined assumptions of health care….
Until next week, we remain yours in primary care.
– Alan, Steve, and Valerie
Primary Care Wednesdays: Pioneering PCPs Offer Insights On Healthcare Reform
I’m really proud to announce the addition of my very first, regular guest blogger team: the healthcare professionals of Doctokr Family Medicine. Each Wednesday they will bring us new insights from the frontier of primary care – their cash-based, high tech, low cost service meets the needs of thousands of local patients at an average yearly cost of only $300. These primary care providers are happy, unhurried, and unfettered by insurance paperwork. They provide 24/7 care by phone, email, office visits and house calls. They’ve negotiated affordable rates with local labs and radiology services and pass on those savings to their patients. Their prices are transparent, affordable, and membership fees are very low.
Sound too good to be true? Well, check back every Wednesday to see what the doctors and nurses of an American primary care revolution have to say.
Their first post will be featured live at 8am, Wednesday, December 3rd.
Left to right: Alan Dappen, M.D.; Steve Simmons, M.D.; Valerie Tinley, N.P.
Bad Science: How To Mislead, Misinform, and Make Mistakes in Medicine
 |
I just finished reading Dr. Ben Goldacre’s new book, Bad Science. It received a very favorable review by the British Medical Journal, and so I thought I’d take a look for myself. After all, I am passionate about patient empowerment and worry sincerely for their safety as healthcare is becoming more and more of a “do-it-yourself” proposition.
Ben is a talented writer – his style is straightforward, accessible, and witty. The premise of the book is to expose the underbelly of science – how it’s miscommunicated to the public (via media, PR, and representatives from the snake oil community) and how research is often poorly designed (by uneducated scientists and government agencies, for-profit pharmaceutical companies, and biased physicians).
The case studies presented in Bad Science are especially poignant. Ben has selected a few shining examples of self-promoting figures who have risen to the highest rank of “expert” in the eyes of the media – all the while referring to themselves as “doctor” and yet only having a Ph.D. from an online correspondence school. Their legal bully tactics, fabrication of data to support their proprietary health gimmick, and extreme narcicism – are excellent studies in poor character triumphing over common sense. It is painful to see how successful snake oil salesmen can be, even in these modern and “enlightened” times.
Bad Science carefully dismantles the pseudoscience that underlies many of the claims of alternative medicine. He clearly demonstrates how research can be manipulated to demonstrate a positive effect for any therapeutic intervention, and explains why cosmetic and nutrition research are particularly rife with false positive results.
Ben also explores the role of the human psyche in misunderstanding science. Our deep desire to find a 1:1 correlation between every cause and effect is difficult to overcome. We want 1) to bring artificial simplicity out of complexity, 2) a quick fix in a pill form, 3) to believe in “breakthrough therapies,”4) to read sensational or scintillating news headlines. Unfortunately, science is often coopted to pander to these wants, rather than illuminate the truth.
Finally, Bad Science explores the many ways that statistics can be manipulated to support any claim. In fact, human intuition about math in general is quite flawed, which works against us as we try to understand the data collected by researchers.
I finished the book feeling enlightened but somewhat despairing – yearning to read a sequel, “Good Science” if only to restore my hope in the idea that wise people will have the courage to seek truth over sensationalism, and value objectivity over subjectivity for the greater good of all.
What does Ben Goldacre think we can do to combat the tidal wave of bad science on the Internet? He suggests that people of sound mind blog about the subject as frequently as possible, so that those who are searching for a voice of reason may find one. I blog here and at sciencebasedmedicine.org for that very purpose.
In my next post, I’ll summarize some tips from Bad Science that will help you to recognize when a health message is likely to be inaccurate.
