October 1st, 2009 by Peter Lipson, M.D. in Better Health Network, Opinion, Quackery Exposed
No Comments »
One of the common themes regarding alternative medicine is the reversal of normal scientific thinking. In science, we must generally accept that we will fail to validate many of our hypotheses. Each of these failures moves us closer to the truth. In alternative medicine, hypotheses function more as fixed beliefs, and there is no study that can invalidate them. No matter how many times a hypothesis fails, the worst that happens is a call for more research.
Sometimes this is the sinister and cynical intent of an alternative practitioner—refuse to let go of a belief or risk having to learn real medicine. Often, though, there are flaws in our way of thinking about data that interfere with our ability to understand them.
Read more »
August 20th, 2009 by Steve Novella, M.D. in Better Health Network, Quackery Exposed
2 Comments »
It is my contention that terms such as “complementary and alternative medicine” and “integrative medicine” exist for two primary purposes. The first is marketing – they are an attempt at rebranding methods that do not meet the usual standards of unqualified “medicine”. The second is a very deliberate and often calculating attempt at creating a double standard.
We already have a standard of care within medicine, and although its application is imperfect its principles are clear – the best available scientific evidence should be used to determine that medical interventions meet a minimum standard of safety and effectiveness. Regulations have largely (although also imperfectly) reflected that principle, as have academia, publishing standards, professional organizations, licensing boards, and product regulation.
With the creation of the new brand of medicine (CAM and integrative) came the opportunity to change the rules of science and medicine to create an alternative standard, one tailor made for those modalities that do not meet existing scientific and even ethical standards for medicine. This manifests in many ways – the NCCAM was created so that these modalities would have an alternate standard for garnering federal dollars for research. Many states now have “health care freedom laws” which create a separate standard of care (actually an elimination of the standard of care) for self-proclaimed “alternative” practices.
But perhaps the most insidious and damaging double-standard that is being advocated under the banner of CAM is a separate standard of scientific research itself. The normal rules of research that have evolved over the last few centuries are being subtly altered or discarded, with clever newspeak. It is a way for proponents to choose their evidence, rather than having the evidence decide what works and what does not work. We saw this strategy at play with the recent acupuncture study for back pain that clearly showed acupuncture was no more effective than placebo acupuncture. Proponents (propagated by an uncritical media) turned scientific logic on its head by interpreting this result as indicating that placebo acupuncture must work also (if only we could figure out how, they unconvincingly mused).
We see this strategy at work also with the use of so-called “pragmatic” studies – a rebranding of “unblinded” studies. This is a way to choose their evidence – in this case, poorly controlled unblinded studies that are more likely to reflect the bias of the researchers and therefore give them a result that they like. This is their reaction to well-designed placebo controlled trials that show their preferred modality does not work.
Another strategy is to change the meaning of the concept of placebo effects. This one was ready-made, and most people grossly misunderstand the nature of “the” placebo effect. One of my first articles for SBM was about the placebo effect because this concept is so criticial to science-based medicine. To summarize – the placebo effect is really many effects. It is everything other than a physiological response to the treatment. It is not all a real effect of mind-over-matter – it includes every bias and artifact of observation as well. It includes things like subjects reporting they feel better to the researcher because they want the treatment to work and they want to please the authority figure, who also wants the treatment to work and may be encouraging the perception of benefit.
It is most important to understand how the term “placebo effect” is used in the context of a controlled clinical trial. Scientific methodology is about controlling variables – because we want to know which variables work and which ones do not. In any clinical scenario there are a multitude of variables that may affect the outcome or the perception of the outcome. Therefore a well-designed study maximally controls all the variables – ideally so that the one variable of interest (the treatment) is completely isolated. This is accomplished in a number of ways. One method is randomization – randomly assigning subjects to the various treatment and placebo arms of a clinical trial. Randomization combined with sufficiently large trial size (number of subjects) results in all variables not specifically controlled for averaging out among the various arms. Another way to look at is that randomization prevents systematic biases in who gets treated and who gets a placebo from affecting the results.
Another method of controlling variables is the double-blind placebo control. Ideally one group of subjects will receive the treatment being studied while another group will receive a treatment that is identical in every way except that it is inert (i.e. it controls for all possible variables and isolated the one variable of interest – the treatment). Both the subject and the examiner are blinded to which is which to control for psychological effects. In order to conclude that the treatment “works” those subject receiving the active treatment must do statistically significantly better than those receiving the placebo. If the activity of the treatment was the only variable, then we can confidently conclude it was responsible for the improvement.
I know this is all very basic, but it is these very basic concepts that are being challenged by proponents of so-called CAM. They are trying to say the the effect measured in the placebo arm of such studies is a real effect, something valuable and alone is sufficient to justify the treatment. This philosophy has been termed by critics “placebo medicine” and is just the latest attempt at creating a double standard. But the claim is utterly ignorant of the scientific nature of the placebo effect. It is a method of controlling for biases, artifacts, and variables (known and unknown) – it is not a real effect.
There may be some non-specific therapeutic effects mixed into placebo effects. For example, people who are being studied tend to take better care of themselves and are more compliant with treatments (because they are being watched). They may also feel better as a result of the positive attention from a health care provider – old-fashioned good bedside manner. These are some of the variables being controlled for. But it is scientifically absurd to argue that they justify an ineffective treatment. But that is exactly what CAM proponents are doing.
The latest manifestation of this strategy is a report put out in the UK by The Kings Fund – a health policy charity. They put together a committee to examine how the UK can find evidence to support CAM therapies. They are not interested in figuring out “if” such treatments work, but rather how they can show “that” they work. They report:
Explaining the need for different types of research when assessing complementary practice, Professor Dame Carol Black said: ‘It has become widely accepted that a stronger evidence base is needed if we are to reach a better understanding of complementary practices and ensure greater confidence in their clinical and cost effectiveness. The challenge is to develop methods of research that allow us to assess the value of an approach that seeks to integrate the physical intervention, the personal context in which it is given, and non-specific effects that together comprise a particular therapy.’
Got that? We need new kinds of research (read “double standard”) in order to demonstrate the value of these special CAM practices. The reason that we need to find new ways to demonstrate their value is because they fail under the accepted scientific methods. The last sentence is just a fancy way of saying that placebo effects should count as real effects.
It further says:
‘As long as findings from research can provide confidence in the positive effect of the physical intervention at the heart of the treatment, then any added benefit brought by the therapeutic relationship and the context for treatment should count as part of the treatment effect,’ the report says.
‘For complementary therapies such a holistic approach to effectiveness should be adopted by bodies such as NICE, when comparing cost-effectiveness across a range of treatments.’
The “physical intervention at the heart of the treatment” is functionally the same thing as – non-specific placebo effects. They want to take a “holistic” approach to evidence (another useful marketing brand), meaning they get to decide what the evidence means. George Orwell would be proud.
As usual, Edzard Ernst (the go to expert for the media) gets it exactly right. He is quoted as saying:
‘This is the introduction of double standards through the back door.’
‘In this case we might as well allow an ineffective medication on the market, because it too will have a placebo effect.’
That latter point is a favorite of mine as well. Whenever CAM proponents try to change the rules of science to suit their needs, I invite my r
*This blog post was originally published at Science-Based Medicine*
