The Faculty of Homeopathy Offer Flawed Answers To Criticisms

September 6, 2009 at 8:17 pm (Homeopathy) (, , )

The Faculty of Homeopathy have released a document purporting to answer criticisms made of the therapy. [A Copy is available here as a PDF.] A blog named “The Great DBH Rant” has dealt with the response to criticism that homeopathy is “totally implausible with no basis in real science”. I decided to take a look at some of the other responses made by the Faculty of Homeopathy.

They say: There is no clinical evidence for the effectiveness of homeopathy. We say: There is evidence – take another look”

Personally, I doubt that critics really do say that there is no clinical evidence for the effectiveness of homeopathy – I usually see the point made that the evidence of efficacy usually relied upon by advocates of homeopathy is poor. Larger, better-designed trials seem to be more likely to be negative than smaller, poorly-designed trials. The Faculty claims that there is a “good body of clinical evidence” and to back up this claim points to there being 138 trials, of which “60 [are] positive, 10 negative and 68 not statistically conclusive”. These numbers are meaningless – we have no indication of how well-designed any of these studies are or of how many trials have gone unpublished. If there were 60 small, poorly-designed trials that were positive and 10 large well-designed trials that were negative then the 60 positive trials could probably be safely ignored. They go on to claim that five out of six meta analyses “show homeopathy to have a greater effect than placebo”, but as AP Gaylard has pointed out in a piece on a recent ‘refutation’ of critics produced by Lionel Milgrom, there is a tendency among homeopaths “to accentuate the positive and ignore the caveats”. The meta analyses cited by Gaylard rightly point out that the evidence is not sufficient and future trials should be well-performed, rigorous, and systematic – which is a rather damning indictment of the current state of the evidence.

They say: We now know homeopathy is no more than placebo
We say: No we don’t! One flawed study does not have the last word

Oh dear, they’re talking about Shang being “flawed”. To support their claims regarding the ‘flaws’ in Shang et al, they cite two papers: Rutten & Stolper and Ludtke & Rutten. I refer you firstly to this: doi:10.1016/j.homp.2008.12.003 – the analysis by Rutten and Stolper contains misconceptions of Shang et al., contains some important errors, and does not show that the Shang et al. study was an invalid analysis. As for Ludtke and Rutten, their conclusion stated that “Our results do neither prove that homeopathic medicines are superior to placebo nor do they prove the opposite…it occurs that Shang’s conclusions are not so definite as they have been reported and discussed.” Their paper may have been titled “The conclusions on the effectiveness of homeopathy highly depend on the set of analyzed trials”, but the conclusions on the effectiveness of homeopathy *do not* highly depend on the set of analyzed trials, if an appropriate test is used. [Note: I’ve borrowed (very) heavily here from the Hawk/Handsaw blog: Rutten posts.]

They say: Homeopaths mislead their patients and give them false hope
We say: Our critics appear to believe the public is gullible: they dismiss patient-reported outcome studies as an example of the placebo response

Patients of homeopaths are not necessarily being characterised as gullible here – we are all placebo responders. The results of patient-reported outcome studies though – whether the Faculty likes it or not – are likely to be susceptible to the placebo effect. I’m not sure which patient-outcome studies the Faculty is referring to here, as no relevant citations of studies into homeopathy are given. The fact is, if homeopaths are telling patients that their remedies can help with their condition and the patient takes this to mean that the remedy is effective above placebo then they are being misled and given false hope – I don’t think that this is being done deliberately, I simply think that homoepaths are themselves misled (after all, “you are the easiest person to fool”; RP Feynman).

They say: Hundreds of millions of pounds are wasted on homeopathy every year – we have seen a figure of £500M quoted!
We say: Homeopathy seems to be cost-effective and could save the NHS money

For homeopathy to be cost-effective it would first have to be effective.

They say: Homeopathy has no place in the NHS
We say: Homeopathy plays a vital role in the NHS by filling ‘effectiveness gaps’ and as a treatment option for patients who cannot take conventional medicines

I do not wholly disagree with this. I think that in theory it is possible that homeopathy could have a place in the NHS – if it were the case that a placebo were helpful for some people with certain (self-limiting, non-serious) conditions. There are, though, a few obstacles for homeopathy to overcome. First, it is possible that people receiving homeopathy on the NHS would see this as a legitimisation of the treatment and would use homeopathy for more serious conditions (e.g. malaria prophylaxis). Secondly, there is an ethical dimension to this – David Colquhoun has written about the “lying dilemma”. Here, Colquhoun writes that: “The whole trend in medicine has been to be more open with the patient and to tell them the truth. To maximise the benefit of alternative medicine, it is necessary to lie to the patient as much as possible” (referring to the need to maximise the placebo effect by omitting to tell the patient that the treatment is inert*).

They say: Homeopaths should not treat serious diseases like cancer
We say: In reality cancer patients are treated with conventional medicine; some also choose homeopathy and other interventions; others seek homeopathy to provide relief from side effects of cancer drugs

Homeopaths should not treat serious diseases like cancer – or Aids, or malaria. The Faculty might prefer to believe that this does not happen, that conventional medicine is always provided to people who choose to use homeopathy for serious diseases, but this is clearly not the case. Take malaria prophylaxis as an example. Then there’s the homeopaths who advise against MMR vaccination, or propose an MP3 remedy for Aids: link.

They say: Homeopathy is alternative, wacky and unregulated
We say: Homeopathy is a complementary medicine that is integrated alongside conventional treatments by medically qualified specialists

Homeopathy is alternative in the sense that it is used as an alternative to real medicine that might actually work. It is wacky in the sense that believers in homeopathy think that a remedy that contains no active ingredient can still work – by somehow rectifying the disturbances in your “vital force” (and I haven’t even mentioned banging remedies on a leather-bound book to ‘potentize’ them). It is unregulated in the sense that there is no proper regulation of homeopaths. Associations seem to claim to regulate members, but their actions do not necessarily accord with their words. Nothing the Faculty says in their response to this criticism refutes the points made – homeopathy is wacky and it is an alternative medicine. Homeopathic doctors are statutorily registered with the General Medical Council – but this is only a specific group of the homeopaths practising in this country and you do not need to be registered with a regulator in order to practise as a homeopath.

More

I haven’t looked at the claim already covered by DBH and I have left the final claim for now – that conventional medicine is not the only medicine that can help patients. That, I think, deserves a blog post of its own. How many conventional treatments are known to be effective? I’ve seen several unreliable estimates and may investigate further in order to see whether there is some reliable information on this.

*Regarding the lying dilemma and maximising the placebo effect by omitting to tell the patient that the treatment is inert: “The primary finding is that patients can be willing to take placebo and can improve despite disclosure of the inert content of the pills” [Lee Crandall Park MD, link, download PDF.]

EDIT, 27th November: since writing this post, I have been reading more about the supposedly powerful placebo and I think that my comments above on placebo are perhaps more positive than the placebo effect warrants.

12 Comments

  1. Twitter Trackbacks for The Faculty of Homeopathy Offer Flawed Answers To Criticisms « Stuff And Nonsense [jdc325.wordpress.com] on Topsy.com said,

    […] The Faculty of Homeopathy Offer Flawed Answers To Criticisms « Stuff And Nonsense jdc325.wordpress.com/2009/09/06/the-faculty-of-homeopathy-offer-flawed-answers-to-criticisms – view page – cached #RSS 2.0 Stuff And Nonsense » The Faculty of Homeopathy Offer Flawed Answers To Criticisms Comments Feed Stuff And Nonsense Better Blogs Everyone’s A — From the page […]

  2. Mojo said,

    There was a previous version of the same document on the FoH website (at the same url). That said that “134 randomised controlled trials (RCTs) have been published to the end of 2007: 59 positive, 8 negative and 67 not statistically conclusive.”

    So that’s four new trials in 2008, one “positive”, one “not statistically conclusive” and two “negative”.

    We can’t be entirely sure exactly what is meant by these terms since they don’t seem to have published anything beyond the bare figures, but I suspect that they are using the same “positive/inconclusive/negative” categorisations as Mathie RT: The research evidence base for homeopathy: a fresh assessment of the literature. Homeopathy (2003) 92, 84–91, which uses similar categories (Dr. Mathie is Research Development Adviser at the British Homeopathic Association & Faculty of Homeopathy). That found, up to the end of 2002, “50 papers [that] report a significant benefit of homeopathy in at least one clinical outcome measure, 41 that fail to discern any inter-group differences, and two that describe an inferior response with homeopathy.” It goes on to describe those two studies as having found homeopathy to have “less effect than placebo”, so it appears that the “negative” studies are those in which homoeopathy was significantly worse, while the ones that were “not statistically conclusive” found no statistically significant effect.

    Comparing the figures given in this paper with their current figures also suggests that since 2002 the results have been considerably worse for homoeopathy than before: 10 positive, 19 with no significant effect, and 8 negative. Perhaps they are now producing false negatives as often as false positives.

    You comment that the figures are pretty meaningless as they don’t tell us anything about the quality of the various studies. The 2003 Mathie paper has something to say about that: “This review deliberately does not categorise published trials in homeopathy by their intrinsic scientific quality, for information on this issue is already available.” It then notes the issues to do with the effect of study size and quality on outcome, and says that “The above caveats (low volume, publication bias, low quality) are important in considering the research evidence base of homeopathy,” but then goes on to say that more than half of these papers of unknown quality show positive effects for homoeopathy, thus effectively ignoring this known issue in its conclusions.

    Another interesting comment regarding homoeopathic research comes from Linde & Melchart: Randomized controlled trials of individualized homeopathy: a state-of-the-art review (J Altern Complement Med. 1998 Winter;4(4):371-88): “The motivation for doing trials seems less to be innovation or self-critical evaluation of performance (which is generally considered to be the motivation for good research) but rather justification in the face of a hostile scientific establishment.” This has implications not only for study quality but also for the possibility of reporting/publication bias. As Feynman said, “you are the easiest person to fool”.

  3. apgaylard said,

    Well argued. I must admit that I’m not very comfortable with the way the homeopathists tend to classify trial outcomes. I keep thinking about this: If a placebo (homeopathy) is tested against another placebo in a fair trial what is the most likely result? I suspect that these kind of results get binned as “not statistically conclusive” rather than negative, which I would argue is the fair assessment.

    The definition Mojo quotes for a ‘positive’ outcome: “a significant benefit of homeopathy in at least one clinical outcome measure” also favours trials that record large numbers of outcome measures. I would argue that only the pre-determined main outcome measures should be assessed.

    As Mojo points out the failure to control for quality makes the conclusions that they are inviting people to draw from this kind of analysis questionable. The lesson from the meta-analyses is that lower quality trials are more likely to show an apparant effect relative to placebo. Shang also shows that size is the key bias.

    Talking about Shang (yawn!), I recon this must be one of the most closely examined studies of homeopathy. Actually, Ludtke & Rutten managed to replicate Shang’s result (putting to rest a number of silly conspiracy theories) and confirm that they had indeed ended up with the set of least biased trials.

    As David Gorski put it(here):

    In other words, if a random effect meta-analysis is used, one can torture marginally significant odds ratios out of the data; if a meta-regression is used, one can’t even manage that! In other words, this study actually shows tht it doesn’t really matter too much which high quality studies are involved, other than that adding lower quality studies to higher quality studies starts to skew the results to seemingly positive values.

    Exactly as one who knows anything about meta-analysis would predict.

    Thanks for the link to my post.

    Good places to start for the numbers on the evidence base for conventional treatments:
    What proportion of healthcare is evidence based?Resource Guide
    How Much Modern Medicine is Evidence-Based

    Admin edit: to fix link to “A Homeopathic Refutation”.

  4. jdc325 said,

    Thank you for an interesting and informative comment Mojo.

  5. zeno said,

    It appears this particular version was published on 19 January 2009 (http://www.facultyofhomeopathy.org/media/news/we_answer_the_critics.html).

    I wonder why it took so long to surface? You’d have thought they would have been shouting it from the pulpits…

  6. zeno said,

    Sorry, 16 January 2009!

  7. jdc325 said,

    @apgaylard: I’m afraid my spam filter mistakenly grabbed your comment, so it hadn’t appeared when I replied to Mojo earlier. Thank you for commenting on this.
    Excellent points re the classification of studies by advocates of homeopathy – agree that ‘negative’ is probably a fair assessment of these ‘inconclusive’ trials.

    Thanks also for the links on the evidence base for conventional medicine.

  8. Mojo said,

    I think the 16th January 2009 document may have been the earlier version I mentioned above (I’m pretty certain it was still there in March).

    Incidentally, both versions of the document state that “Dr Peter Fisher, Clinical Director of the Royal London Homeopathic Hospital, has written a detailed commentary on these two studies”, but the link they give just takes me to a log-in page for the Journal of Clinical Epidemiology. The two studies are the Ludtke/Rutten and Rutten/Stolper papers from last October, but I haven’t managed to find anything by Peter Fisher in the last couple of years-worth of the journal. Does anyone have access to a subscription for this journal so they can see what the link leads to?

  9. N M said,

    Homeopathy cures even when Conventional Allopathic Medicine (CAM) fails

  10. jdc325 said,

    Hi Nancy – I see that you are still leaving your traditional drive-by comment on anything that mentions homeopathy. How is the Think Humanism challenge going, by the way? Or are you ducking that one?

  11. Zeno said,

    Nancy

    We’re STILL waiting for you at Think Humanism so we can do the test of a homeopathic potion you promised you’d do (and which you claimed you could pass).

    Are you EVER going to keep your promise – or do you now think you can’t tell the difference between a homeopathic potion and water?

  12. Zeno said,

    Snap! Cross posted with jdc.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: