Someone named Lex commented on Dr Briffa’s statistical significance post and made this remark: “when it comes to homeopathy and food supplements they always do include the placebo effect- they would run a mile from a double-blind trial” and Briffa responded with this:
I entirely understand the need of some people to perform randomised, placebo controlled trials (chiefly, in an effort to discern whether what is being tested has a ‘real’ effect or not). However, in the real world (that’s real people, with real problems) the fact that the placebo response may account for a lot of even the whole of a clinical response is not generally important for those the treatment is intended to help (those real people with real problems, again).
You tell us that ‘they’ (whoever ‘they’ are) in homeopathy and food supplements would ‘run a mile from a double-blind trial’. I can’t speak for homeopathy (as I don’t know much about it), but you are simply incorrect with regard to double-blind studies on nutritional agents.
I tried to point out that while Briffa had posted a link (to just one double-blind study on fish oil), the general point that Lex made was valid – most food supplement and homeopathy firms do not do double-blind trials on their products. After all, they aren’t required by regulation to do so and if they took the chance and conducted a study, then the results might well be, shall we say, ‘not to their liking’ and the money they have spent on this scientific study would be considered to have been wasted. I also pointed out that there were other issues surrounding CAM research such as poorly designed [non-] trials and companies wanting control over data. Briffa responded by claiming that Lex’s point was absolute and that Lex had claimed all supplement/homeopathy firms would run a mile from double-blind tests. Briffa seems to have a habit of making everything absolute rather than relative in order to make a point. He complains that I use qualifying terms when making my claims (‘Lex wrote ‘they would run a mile from a double-blind trial’. This is stated in absolute terms, no? Lex did not use words such as ‘generally’, or ‘usually’ or ‘tend to’. No, they – all of them – would not engage in double-blind research is the assertion’) and then tries to claim that his single example proves Lex’s point to be false – believing that he does so in the manner of Karl Popper pointing out that “if the hypothesis is that there are only white swans, I only need to show the existence of one black swan (not two, or 10 or 100) to disprove the hypothesis” – and that I have fallen into some cleverly constructed trap. The trap is that by citing one study, Briffa could lure some unsuspecting ‘scientist’ or ‘academic’ [sorry – the pejorative ‘scare quote’ thing seems to be contagious] into complaining that he had only cited one study. Unfortunately for John, I used appropriate qualifiers in my remarks about supplement/homeopathy firms so I don’t think I did fall into his ‘cleverly-constructed trap’. I simply pointed out that Lex’s argument had some validity – I never claimed that all supplement/homeopathy firms ran from double-blind tests. Also unfortunate for John is that in his crowing about getting the scalp of a ‘scientist’ or ‘academic’, he forgot to ask if I actually was either of these things. Assumption may be good enough for Dr John, but in actual fact I am neither an academic nor a scientist – he’s caught a layman [it must be a bit like going big-game hunting and shooting a ferret]. I’ll let John show just how clever he is – he can have the final say in this blog post:
Lex wrote ‘they would run a mile from a double-blind trial’. This is stated in absolute terms, no? Lex did not use words such as ‘generally’, or ‘usually’ or ‘tend to’. No, they – all of them – would not engage in double-blind research is the assertion.
Now, as I said, there is a significant body of double-blind research in the area (some of which is industry-funded, of course). I actually started out with a list of studies to rebut Lex’s claim. But then I remembered something I think Karl Popper said about swans: If the hypothesis is that there are only white swans, I only need to show the existence of one black swan (not two, or 10 or 100) to disprove the hypothesis.
And then I wondered if someone with a very pro-science stance would ‘take the bait’, focus on the fact that I only presented one study, and in so doing would betray ‘good science’ (and even logic) and demonstrate the subjectivity and prejudice that is so often found in science and academia.