Bad Science: How homeopaths misrepresent the evidence

A favorite methodology amongst cranks and woo-meisters is to take legitimate scientific research, and distort or misrepresent its findings until they suit the crank’s presumptions and belief system. And since we all know that Homeopathy is the biggest branch of woo, let’s see how they torture scientific evidence using a huge manufacturer as a use case: Boiron.

Boiron is one of the biggest manufacturers and distributors of homeopathic remedies. It is a French company with a yearly sales turnout of more than £200 million. Nevertheless, they only spend around £2.5 million for research and around £47 million goes to marketing (for a ratio of 1:18!). To put this in perspective, the Big Pharma (so often accused of too much advertising) spends half as much in research than in marketing (1:2).

Obviously, Boiron has a marked interest in presenting homeopathy as a legitimate modality backed up with lots of scientific evidence. Of course the evidence is weak at the best case, totally damning at the worst! Cherry picking is a favorite practise of pseudoscientists, where they pick only the studies that provide positive results, forgetting those that are negative. Further, they never really pay attention to the methodological design of the trial in question. Not all trials are equal: some are methodologically very strict (randomized, placebo controlled, double blinding etc.) while others are weak (small numbers, not adequate randomization etc.).

Boiron implements the usual cherry picking in their Homeopathic Research page. However, this is quite easy to debunk since this is exactly what meta-analysis do: gather all related clinical trials in the field of interest, “sort” them by quality, potentially handle other variables like publication bias, and spit out the overall results in a big table!

Interestingly enough, Boiron discusses the meta-analyses as well. They do so, however, from a very weird perspective, a sort of parallel universe so to say. Somehow, in this Boiron’s universe, the same damning meta-analyses are now actually conclusively positive for homeopathy! Let’s see what’s going on here…

In less than one decade, the conclusions from meta-analyses involving homeopathy have convinced their authors of the acceptability of homeopathic clinical trials and their positive results.

Here Boiron is just preparing the unsuspecting reader of what is to follow. This statement is clearly false and criminally deceptive.

In 1991, a meta-analysis was published which involved the meticulous study of 107 trials. As regards methods used for the clinical evaluation, the conclusion was clear: “it is wrong to say that homeopathy has not been evaluated according to the modern method of controlled trials”. Among these clinical trials, a large majority (81 to be exact) has had positive results concerning the efficacy of homeopathic treatment.

This is J Kleijnen et al “Clinical trials of homoeopathy” review published in the British Medical Journal in 1991 [1]. Boiron conveniently bypasses the conclusions of the authors about the efficacy of homeopathy sticking instead on the side-conclusion that homeopathy is properly evaluated in trials. No matter from which perspective you see this, Boiron has it wrong: many homeopaths often mention that modern clinical trials are not suitable for testing homeopathy because it is an individualized treatment. Boiron however, seems to like the proper evaluation of homeopathy. Of course this is because Boiron sells homeopathic remedies by the kilo! Homeopaths are such a messed-up bunch of deluded pseudoscientists that they cannot even make up their minds on anything…

Second, Boiron manages to cherry-pick on a meta-analysis as well! It doesn’t matter if the majority of the original trials were positive! The point of the meta-analysis is to sort them out based on their quality and try to come up with an overall conclusion. And this is what the authors’ overall conclusion was:

The results of the review may be complicated by publication bias, especially in such a controversial subject as homoeopathy. At the moment the evidence of clinical trials is positive but not sufficient to draw definitive conclusions because most trials are of low methodological quality and because of the unknown role of publication bias.

OK, low quality, publication bias, no definitive conclusions, you get the point. And this is the most favorable meta-analysis you will see! And, importantly, the oldest one! It’s all downhill from now on for the homeopaths… But Boiron continues unfazed:

A similar study was conducted in 1996 at the European Parliament’s request2 . It examined the data from trials involving the efficacy of homeopathic medicines in relation to a placebo or to no treatment. Comparisons were retained satisfying all of the experts’ requirements, who finally concluded that “the number of significant results was clearly not down to chance”.

This is a report by Boissel et al in 1996 [2]. Unfortunately, I could not get hold of this report. I was only able to find a quote from the report which mentioned that: “after examining 184 reports of controlled trials, only 17 were worth considering” and concluded: “the number of participants was too small to draw any conclusions about the effectiveness of homoeopathic remedies for any specific condition“.

And we now move on to the hottest part of this post: the 1997 Linde et al. meta-analysis [3]. This is the one that homeopaths love to cite, as it showed positive results above placebo! Here is what Boiron (and other homeopaths) mentions:

A year later (1997), a study involving the analysis of 89 trials was published taking close account of the criteria specific to the homeopathic therapeutic method. It concluded that even if evidence of the complete efficacy of homeopathy in the treatment of a single given patient were insufficient, “it was impossible that the clinical effects of the homeopathy were exclusively caused by a placebo effect”.

And here is the conclusion of the Linde paper appearing in Lancet [3]:

The results of our meta-analysis are not compatible with the hypothesis that the clinical effects of homoeopathy are completely due to placebo. However, we found insufficient evidence from these studies that homoeopathy is clearly efficacious for any single clinical condition.

Wow! It seems that homeopathy is indeed an efficient treatment (disregarding Linde’s remarks about insufficient evidence)! Errr, not really. What you will not hear Boiron or any homeopath mention, is the continued research by Linde on this issue. First, in 1998, he published another review of 32 trials that met his criteria, in the Journal of Alternative and Complementary Medicine [4] that concluded:

The results of the available randomized trials suggest that individualized homeopathy has an effect over placebo. The evidence, however, is not convincing because of methodological shortcomings and inconsistencies

We see once again (just like in his 1997 review) that Linde immediately qualifies his interpretation that homeopathy has an effect over placebo, by mentioning the shortcomings of the trials. And it is the effect of these shortcomings that Linde set out to measure, putting the final nail in the homeopathic coffin in 1999.

Being interested in the influence that methodological design and experimental protocol has on homeopathic research, his team published a study of the effects of the quality of clinical trials in homeopathy on the results. For this study they used the same 89 trials of the 1997 review. And this is what they had to say in the Journal of Clinical Epidemiology [5]:

The evidence of bias weakens the findings of our original meta-analysis [7]. Since we completed our literature search in 1995, a considerable number of new homeopathy trials have been published. The fact that a number of the new high-quality trials (e.g. [14,15]) have negative results, and a recent update of our review for the most “original” subtype of homeopathy (classical or individualized homeopathy [16]), seem to confirm the finding that more rigorous trials have less-promising results. It seems, therefore, likely that our meta-analysis [7] at least overestimated the effects of homeopathic treatments.

Note that reference number 7, is Linde et al. 1997 meta-analysis! But is any homeopath willing to listen to the evidence? In short: NO. They never listen. They have a dogmatic belief in homeopathy and no matter what amount of evidence you show them they will simply ignore it. Even Peter Fisher, homeopath to the Queen, makes the same mistake with Linde’s 1997 research in a letter criticizing Ben Goldacre‘s “Benefits and risks of homoeopathy” article in Lancet [6]. To the public’s harm of course.

In any case, it is fun to thoroughly destroy homeopathy, so let’s move on to more Boiron claims! This is arguably the funniest one:

In August 2005, the weekly journal The Lancet published a new study on the effectiveness of homeopathy. In its editorial the journal drew some surprisingly controversial and unfavourable conclusions on homeopathy. It concerns an analysis which like the 3 previous meta-analyses concludes that homeopathic medicine is effective. However, to arrive at the opposite conclusion, the authors implicitly removed series of trials afterwards, retaining just 14 (8 on homeopathy) of the 220 initial trials (110 of which were on homeopathic medicine).

This clearly demonstrates a) ignorance of the science behind a meta-analysis or b) deception of the highest level! This quote refers to the latest meta-analysis by Shang et al. published in the Lancet in 2005 [7]. The last rites of homeopathy indeed. Shang did not “implicitly remove” series of trials afterwards: they used all the trials in their initial analysis to define which ones are the highest quality trials. The surviving 8 homeopathic trials were simply the ones that made the cut. The rest were just not good enough. And of course the conclusion Shang et al. draw is ferocious:

Biases are present in placebo-controlled trials of both homoeopathy and conventional medicine. When account was taken for these biases in the analysis, there was weak evidence for a specific effect of homoeopathic remedies, but strong evidence for specific effects of conventional interventions. This finding is compatible with the notion that the clinical effects of homoeopathy are placebo effects.

And here is another ironic twist in this whole story: Linde send a letter to the Lancet congratulating Shang et al. for their meta-analysis and stating his regret that his 1997 study is being misused by homeopaths!

We congratulate Aijing Shang and colleagues on their meta-analysis examining the clinical effects of homoeopathy. Their methods largely reproduce those of our meta-analysis [note: the 1997 one] on the same topic published in The Lancet 8 years ago. We agree that homoeopathy is highly implausible and that the evidence from placebo-controlled trials is not robust.


Our 1997 meta-analysis has unfortunately been misused by homoeopaths as evidence that their therapy is proven.

Yet homeopathy still remains in fashion… And imagine that Boiron left out (intentionally?) two other meta-analyses, Cucherat et al. published in the European Journal of Clinical Pharmacology in 2000 [8] which concludes:

There is some evidence that homeopathic treatments are more effective than placebo; however, the strength of this evidence is low because of the low methodological quality of the trials. Studies of high methodological quality were more likely to be negative than the lower quality studies

and Ernst et al. review of reviews published in the British Journal of Clinical Pharmacology in 2002 [9] which states:

Electronic databases were searched for systematic reviews/meta-analysis on the subject. Seventeen articles fulfilled the inclusion/exclusion criteria. Six of them related to re-analyses of one landmark meta-analysis. Collectively they implied that the overall positive result of this meta-analysis is not supported by a critical analysis of the data. Eleven independent systematic reviews were located. Collectively they failed to provide strong evidence in favour of homeopathy. In particular, there was no condition which responds convincingly better to homeopathic treatment than to placebo or other control interventions. Similarly, there was no homeopathic remedy that was demonstrated to yield clinical effects that are convincingly different from placebo. It is concluded that the best clinical evidence for homeopathy available to date does not warrant positive recommendations for its use in clinical practice.

Notice that the “landmark study” which is mentioned is Linde’s 1997 study [3]!

However, despite all the evidence that homeopathy does not work better than placebo (especially considering that all positive results were on the verge of significance, although homeopaths make some pretty amazing claims for miraculous cures!) Boiron manages to reach this mind-boggling conclusion:

These meta-analyses highlight the perfectibility of the research conducted in homeopathy and the necessity of rigorously and systematically pursuing them. But they also show that the reliability of the testing to which homeopathy is already subjected is beyond doubt.

Science has been ruthless in deciding that homeopathy (or acupuncture, reiki, etc. for what matters) does not work as advertised. In fact, it does not work any better than a simple sugar pill. Yet homeopaths still make the same claims and still drive unsuspecting and scientifically illiterate people away from evidence-based medicine and proper treatments. This is not harmless as many apologists would claim. It has dire consequences and can lead to totally unnecessary deaths.

Bottom line: Boiron, one of the biggest (the biggest?) players in the homeopathic industry, fails miserably to support their case, and manages only to enhance the notion that con men and charlatans must resort to disingenuous means in order to promote their woo-woo.

  1. J Kleijnen, P Knipschild and G ter Riet, Clinical trials of homoeopathy, BMJ 302 (1991), pp. 316–323 []
  2. JP Boissel, M Cucherat, M Haugh and E Gauthier, Critical literature review on the effectiveness of homoeopathy: overview of data from homoeopathic medicine trials, Homoeopathic Medicine Research Group. Report to the European Commission, Brussels, Belgium (1996), pp. 195–210 []
  3. K Linde, N Clausius and G Ramirez, Are the clinical effects of homoeopathy placebo effects? A meta-analysis of placebo-controlled trials, Lancet 350 (1997), pp. 834–843 [] [] []
  4. K Linde and D Melchart, Randomized controlled trials of individualized homeopathy: a state-of-the-art review, J Alter Complement Med 4 (1998), pp. 371–388 []
  5. K Linde, M Scholz, G Ramirez and N Clausius, Impact of study quality on outcome in placebo-controlled trials of homeopathy, J Clin Epidemiol 52 (1999), pp. 631–636 []
  6. B Goldacre, Benefits and risks of homoeopathy, Lancet 370 (2007), pp. 1672–1673 []
  7. A Shang, K Huwiler-Müntener and L Nartey et al., Are the clinical effects of homoeopathy placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy, Lancet 366 (2005), pp. 726–732 []
  8. M Cucherat, MC Haugh, M Gooch and JP Boissel, Evidence of clinical efficacy of homeopathy: a meta-analysis of clinical trials, Eur J Clin Pharmacol 56 (2000), pp. 27–33 []
  9. E Ernst, A systematic review of systematic reviews of homeopathy, Br J Clin Pharmacol 54 (2002), pp. 577–582 []