Better Health: Smart Health Commentary Better Health (TM): smart health commentary

Article Comments

Consumer Reports Promotes Alternative Medicine With Questionable Research

Ever since I was a teenager, I’ve intermittently read Consumer Reports, relying on it for guidance in all manner of purchase decisions. CR has been known for rigorous testing of all manner of consumer products and the rating of various services, arriving at its rankings through a systematic testing method that, while not necessarily bulletproof, has been far more organized and consistent than most other ranking systems. True, I haven’t always agreed with CR’s rankings of products and services about which I know a lot, but at the very least CR has often made me think about how much of my assessments are based on objective measures and how much on subjective measures.

Until now.

I just saw something yesterday on the CR website that has made me wonder just how scientific CR’s testing methods are, as CR has apparently decided to promote alternative medicine modalities by “assessing” them in an utterly scientifically ignorant manner. Maybe I just haven’t been following CR regularly for a while, but if there’s an article that demonstrates exactly why consumer product testing organizations should not be testing medical treatments; they are ill-equipped to do so and lack the expertise and knowledge. The first red flag was the title, namely Hands-on, mind-body therapies beat supplements. The second red flag was the introduction to the article:

A new survey of subscribers to Consumer Reports found that prescription drugs generally performed better than alternative therapies for 12 common health problems. But hands-on treatments such as chiropractic care and deep-tissue massage, as well as mind-body therapies such as yoga and meditation, held their own, especially for certain conditions. Far fewer said that dietary supplements helped a lot.

Prescription drugs helped the most for nine of the conditions we asked about: allergies, anxiety, colds and flu, depression, digestive problems, headache and migraine, insomnia, irritable bowel syndrome, and osteoarthritis.

But chiropractic care performed better than drugs for back pain, and deep-tissue massage beat drugs for neck pain. Massage was as also as good as drugs for fibromyalgia. Those hands-on therapies also scored near the top for osteoarthritis as well as for headaches and migraines.

Whatever rigorous testing methodologies CR might bring to various products, its editors clearly have zero clue when it comes to science- and evidence-based medicine if they think that a survey is the appropriate way to determine which treatments work or how well treatments work relative to each other. There was a perfect example of what that is so just the other day with the study that appeared in New England Journal of Medicine that was touted as evidence that the placebo effect is powerful but in reality what the study showed is that, while placebos can make patients think they feel better, they don’t actually do anything to change the underlying pathology of the disease for the better. That’s why science and randomized clinical trials are necessary to determine what therapies work, which ones do not, and which ones work better than others. Surveys are a notoriously unreliable and deceptive (as in self-deceptive) way of trying to assess the relative merits of various therapies, representing as they do, mainly an aggregation of testimonials. Yet that’s what CR is using to try to rank alternative medicine therapies.

Worse, CR concludes its introduction:

For details, see our full report on alternative therapies, including advice on how to find a good chiropractor, massage therapist, yoga instructor, or other alternative-medicine practitioner.

I would submit to you that any reputable testing organization should not–I repeat, should not–be providing advice on how to find alternative medicine practitioners. On the other hand, note the bait and switch. Massage therapy is not necessarily “alternative.” At least it’s not alternative until it’s infused with woo like talk of “life energies” and such. Ditto yoga instructors, given that yoga, stripped of its woo, is basically stretching exercises. As for chiropractors, my standards for what would constitute a “good” chiropractor would be a bit different than most; I’d choose chiropractors who function primarily as physical therapists, eschewing any suggestion that they can cure any disease or treat anything other than musculoskeletal complaints. Any chiropractor who still believes in those mystical, magical “subluxations” that only chiropractors can find or who promote the idea of “innate intelligence” would not be a “good” chiropractor in my book; he’d be a quack.

My scientific orientation aside, it is nonetheless rather interesting to peruse CR’s report on “alternative” treatments entitled Alternative treatments More than 45,000 readers tell up what helped. The first thing I noticed was a rather obvious logical fallacy in the form of argumentum ad populum (i.e., appeal to popularity):

Done anything alternative lately? If so, you have a lot of company. When we surveyed 45,601 Consumer Reports subscribers online, we found that three out of four were using some form of alternative therapy for their general health. More than 38 million adults make in excess of 300 million visits to acupuncturists, chiropractors, massage therapists, and other complementary and alternative practitioners each year in the United States.

One wonders how long it will be before quacks start quoting the figure of three quarters of CR readers using alternative medicine or subtly misrepresenting the figure as three-quarters of Americans. After all, “complementary and alternative medicine” (CAM) and “integrative medicine” (IM) practitioners try very, very hard to paint those of us who are skeptical of their claims as hopeless Luddites trying to resist the inevitable CAM wave washing over medicine. Argumentum ad populum is one of the strongest logical fallacies that CAM proponents use. In fact, CR virtually admits that its survey is utterly useless for telling its readers what does and doesn’t work (although no doubt that’s not how the editors see it) when it states:

A total of 30,332 survey respondents gave us their perceptions of the helpfulness of treatments for their most bothersome conditions over the past two years. The respondents were Consumer Reports subscribers, and our findings might not be representative of the general population. Respondents based their opinions on personal experience, so the results can’t be compared with scientific clinical trials. And our results do not take into account the power of the placebo effect, the tendency of people to find even simulated or sham interventions helpful.

None of these caveats stops CR, though, which bravely dives right into the pool of pseudoscience promoted by appeals to popularity.

Even though this survey is pretty much useless (as CR admits while carrying it out anyway) for providing guidance about what therapies to use, I do have to admit–grudgingly–that this article does provide some rather interesting information, chief of which is that its results show that those nasty, evil, reductionistic prescription drugs in general outperformed any alternative therapy, even in a subjective, self-reported, Internet survey of CR’s subscribers like this. For instance, for allergy, prescription medications and over-the-counter medications were listed by CR readers by far as the top two therapies that “helped a lot.” It’s also interesting to note that only 2% of its readers used chiropractic to treat allergies, although 41% said that it “helped a lot.” Surprisingly, even for conditions for which “mind-body” therapies might be considered effective, prescription medications ruled the roost, including depression and anxiety. Other favorite conditions for which alt-med is used and for which alt-med claims success even though its success is virtually all placebo response yielded to the power of big pharma: Irritable bowel syndrome, insomnia, headache, and colds and flu.

In fact, for only two conditions did any “alternative” modality beat or even come close to conventional medicine even in this biased, self-reported survey: back pain/neck pain (which I lumped together because they’re both the spine), osteoarthritis, and fibromyalgia. And what was effective? For neck/back pain, chiropractic and massage were reported to be as effective as or more effective than prescription medication, which is not surprising because conventional medicine prescribes physical therapy for these conditions anyway and for spine problems chiropractic is generally physical therapy with woo liberally sprinkled on top. For fibromyalgia deep tissue massage was in a dead heat with prescription medication, which is probably more an indication of the lack of good therapies for fibromyalgia right now than it is an indication that alternative therapies work. Ditto osteoarthritis, where the effectiveness of massage probably indicates the same thing.

There are three other results of this survey that are worth mentioning, for instance this passage:

For most conditions we asked about, the No. 1 reason respondents gave for choosing an alternative treatment was simply that they were “a proponent” of it.

“Some people use these therapies because it’s just the way they were raised,” says Richard Nahin, Ph.D., M.P.H., senior adviser for scientific coordination and outreach at the National Institutes of Health’s National Center for Complementary and Alternative Medicine.

Some say they have gone through a transformational process, such as a major illness that has caused them to look at their life in a different way, Nahin says. Others believe dietary supplements are safer than prescription medication because they’re natural, even though that’s not necessarily the case, he says.

In other words, CAM is belief-based medicine, not science-based medicine, and here we appear to have a senior advisor from NCCAM admitting just that.

The second thing is that this article once again tries to make the claim that conventional, scientific medicine is “embracing” CAM by including a section discussing doctors and CAM. Of course, the interesting thing about this section is that it actually portrays physicians who practice CAM as being relatively uncommon, while representing one as being a “brave maverick doctor” who practices acupuncture. Even though I’ve seen doctors who practice acupuncture before (I’m talking to you, Brian Berman), I still can’t figure out the mental contortions and cognitive dissonance that must be necessary to make that happen. Still, in this article, we have a family practitioner named Dr. Rick Hobbs sticking needles into people to realign their qi. He’s even a kindly-appearing, bespectacled old guy wearing a bow tie! Can it get any more Norman Rockwell than that? Not without extreme difficulty or resurrecting Norman Rockwell himself to paint something.

Finally, CR’s Hands-on and mind-body therapies: A user’s guide is depressing to behold. First, one notes the classic bait and switch, where therapies that could be considered part of science-based medicine, such as massage, yoga (which is just gentle stretching exercise), and meditation (which is more or less relaxation) are represented as “alternative” and then lumped in with quackery (acupuncture) as though the quackery were equivalent. Chiropractic itself would be a form of physical therapy if it were stripped of its vitalistic woo. In other words, not only has CR apparently decided that pointless Internet surveys where the respondents are all self-selected are a valid way to assess the efficacy of therapies, even while in essence admitting that they are not, but it’s apparently bought into the pseudoscientific world view at the heart of CAM. Whether it did so through belief or cynically in order to pander to a large, woo-loving segment of its audience, I don’t know and don’t care. What I do know is that CR has in this article utterly failed its readers and betrayed its history of rigorous product testing.

I hope it was worth it for CR to shoot its own credibility in the foot with such abandon.

*This blog post was originally published at Science-Based Medicine*


You may also like these posts

Read comments »


Comments are closed.

Return to article »

Latest Interviews

How To Make Inpatient Medical Practice Fun Again: Try Locum Tenens Work

It s no secret that most physicians are unhappy with the way things are going in healthcare. Surveys report high levels of job dissatisfaction burn out and even suicide. In fact some believe that up to a third of the US physician work force is planning to leave the profession…

Read more »

Caring For Winter Olympians In Sochi: An Interview With Team USA’s Chief Medical Officer Dr. Gloria Beim

I am a huge fan of the winter Olympics partly because I grew up in Canada where most kids can ski and skate before they can run and partly because I used to participate in Downhill ski racing. Now that I m a rehab physician with a reconstructed knee I…

Read more »

See all interviews »

Latest Cartoon

Richmond, VA – In an effort to simplify inpatient medical billing, one area hospitalist group has determined that “altered mental status” (ICD-9 780.97) is the most efficient code for use in any patient work up.

“When you enter a hospital, you’re bound to have some kind of mental status change,” said Dr. Fishbinder, co-partner of Area Hospitalists, PLLC. “Whether it’s confusion about where your room is located in relationship to the visitor’s parking structure, frustration with being woken up every hour or two to check your vital signs, or just plain old fatigue from being sick, you are not thinking as clearly as before you were admitted. And that’s all the justification we need to order anything from drug and toxin screens, to blood cultures, brain MRIs, tagged red blood cell nuclear scans, or cardiac Holter monitoring. There really is no limit to what we can pursue with our tests.”

Common causes of mental status changes in the elderly include medicine-induced cognitive side effects, disorientation due to disruption in daily routines, age-related memory impairment, and urinary tract infections.

“The urinalysis is not a very exciting medical test,” stated Dr. Fishbinder. “It doesn’t matter that it’s cheap, fast, and most likely to provide an explanation for strange behavior in hospitalized patients. It’s really not as elegant as the testing involved in a chronic anemia or metabolic encephalopathy work up. I keep it in my back pocket in case all other tests are negative, including brain MRIs and PET scans.”

Nursing staff at Richmond Medical Hospital report that efforts to inform hospitalists about foul smelling urine have generally fallen on deaf ears. “I have tried to tell the hospitalists about cloudy or bloody urine that I see in patients who are undergoing extensive work ups for mental status changes,” reports nurse Sandy Anderson. “But they insist that ‘all urine smells bad’ and it’s really more of a red herring.”

Another nurse reports that delay in diagnosing urinary tract infections (while patients are scheduled for brain MRIs, nuclear scans, and biopsies) can lead to worsening symptoms which accelerate and expand testing. “Some of my patients are transferred to the ICU during the altered mental status work up,” states nurse Anita Misra. “The doctors seem to be very excited about the additional technology available to them in the intensive care setting. Between the central line placement, arterial blood gasses, and vast array of IV fluid and medication options, urosepsis is really an excellent entré into a whole new level of care.”

“As far as medicine-induced mental status changes are concerned,” added Dr. Fishbinder, “We’ve never seen a single case in the past 10 years. Today’s patients are incredibly resilient and can tolerate mixes of opioids, anti-depressants, anti-histamines, and benzodiazepines without any difficulty. We know this because most patients have been prescribed these cocktails and have been taking them for years.”

Patient family members have expressed gratitude for Dr. Fishbinder’s diagnostic process, and report that they are very pleased that he is doing everything in his power to “get to the bottom” of why their loved one isn’t as sharp as they used to be.

“I thought my mom was acting strange ever since she started taking stronger pain medicine for her arthritis,” says Nelly Hurtong, the daughter of one of Dr. Fishbinder’s inpatients. “But now I see that there are deeper reasons for her ‘altered mental status’ thanks to the brain MRI that showed some mild generalized atrophy.”

Hospital administrators praise Dr. Fishbinder as one of their top physicians. “He will do whatever it takes to figure out the true cause of patients’ cognitive impairments.” Says CEO, Daniel Griffiths. “And not only is that good medicine, it is great for our Press Ganey scores and our bottom line.”

As for the nursing staff, Griffiths offered a less glowing review. “It’s unfortunate that our nurses seem preoccupied with urine testing and medication reconciliation. I think it might be time for us to mandate further training to help them appreciate more of the medical nuances inherent in quality patient care.”

Dr. Fishbinder is in the process of creating a half-day seminar on ‘altered mental status in the inpatient setting,’ offering CME credits to physicians who enroll. Richmond Medical Hospital intends to sponsor Dr. Fishbinder’s course, and franchise it to other hospitals in the state, and ultimately nationally.

***

Click here for a musical take on over-testing.

See all cartoons »

Latest Book Reviews

The Spirit Of The Place: Samuel Shem’s New Book May Depress You

When I was in medical school I read Samuel Shem s House Of God as a right of passage. At the time I found it to be a cynical yet eerily accurate portrayal of the underbelly of academic medicine. I gained comfort from its gallows humor and it made me…

Read more »

Eat To Save Your Life: Another Half-True Diet Book

I am hesitant to review diet books because they are so often a tangled mess of fact and fiction. Teasing out their truth from falsehood is about as exhausting as delousing a long-haired elementary school student. However after being approached by the authors’ PR agency with the promise of a…

Read more »

Unaccountable: A Book About The Underbelly Of Hospital Care

I met Dr. Marty Makary over lunch at Founding Farmers restaurant in DC about three years ago. We had an animated conversation about hospital safety the potential contribution of checklists to reducing medical errors and his upcoming book about the need for more transparency in the healthcare system. Marty was…

Read more »

See all book reviews »