OK, Just a Little Bit More Briffa
Someone named Lex commented on Dr Briffa’s statistical significance post and made this remark: “when it comes to homeopathy and food supplements they always do include the placebo effect- they would run a mile from a double-blind trial” and Briffa responded with this:
I entirely understand the need of some people to perform randomised, placebo controlled trials (chiefly, in an effort to discern whether what is being tested has a ‘real’ effect or not). However, in the real world (that’s real people, with real problems) the fact that the placebo response may account for a lot of even the whole of a clinical response is not generally important for those the treatment is intended to help (those real people with real problems, again).
And this:
You tell us that ‘they’ (whoever ‘they’ are) in homeopathy and food supplements would ‘run a mile from a double-blind trial’. I can’t speak for homeopathy (as I don’t know much about it), but you are simply incorrect with regard to double-blind studies on nutritional agents.
I tried to point out that while Briffa had posted a link (to just one double-blind study on fish oil), the general point that Lex made was valid – most food supplement and homeopathy firms do not do double-blind trials on their products. After all, they aren’t required by regulation to do so and if they took the chance and conducted a study, then the results might well be, shall we say, ‘not to their liking’ and the money they have spent on this scientific study would be considered to have been wasted. I also pointed out that there were other issues surrounding CAM research such as poorly designed [non-] trials and companies wanting control over data. Briffa responded by claiming that Lex’s point was absolute and that Lex had claimed all supplement/homeopathy firms would run a mile from double-blind tests. Briffa seems to have a habit of making everything absolute rather than relative in order to make a point. He complains that I use qualifying terms when making my claims (‘Lex wrote ‘they would run a mile from a double-blind trial’. This is stated in absolute terms, no? Lex did not use words such as ‘generally’, or ‘usually’ or ‘tend to’. No, they – all of them – would not engage in double-blind research is the assertion’) and then tries to claim that his single example proves Lex’s point to be false – believing that he does so in the manner of Karl Popper pointing out that “if the hypothesis is that there are only white swans, I only need to show the existence of one black swan (not two, or 10 or 100) to disprove the hypothesis” – and that I have fallen into some cleverly constructed trap. The trap is that by citing one study, Briffa could lure some unsuspecting ‘scientist’ or ‘academic’ [sorry – the pejorative ‘scare quote’ thing seems to be contagious] into complaining that he had only cited one study. Unfortunately for John, I used appropriate qualifiers in my remarks about supplement/homeopathy firms so I don’t think I did fall into his ‘cleverly-constructed trap’. I simply pointed out that Lex’s argument had some validity – I never claimed that all supplement/homeopathy firms ran from double-blind tests. Also unfortunate for John is that in his crowing about getting the scalp of a ‘scientist’ or ‘academic’, he forgot to ask if I actually was either of these things. Assumption may be good enough for Dr John, but in actual fact I am neither an academic nor a scientist – he’s caught a layman [it must be a bit like going big-game hunting and shooting a ferret]. I’ll let John show just how clever he is – he can have the final say in this blog post:
Lex wrote ‘they would run a mile from a double-blind trial’. This is stated in absolute terms, no? Lex did not use words such as ‘generally’, or ‘usually’ or ‘tend to’. No, they – all of them – would not engage in double-blind research is the assertion.
Now, as I said, there is a significant body of double-blind research in the area (some of which is industry-funded, of course). I actually started out with a list of studies to rebut Lex’s claim. But then I remembered something I think Karl Popper said about swans: If the hypothesis is that there are only white swans, I only need to show the existence of one black swan (not two, or 10 or 100) to disprove the hypothesis.
And then I wondered if someone with a very pro-science stance would ‘take the bait’, focus on the fact that I only presented one study, and in so doing would betray ‘good science’ (and even logic) and demonstrate the subjectivity and prejudice that is so often found in science and academia.
Stuff And Nonsense 
John Briffa said,
May 31, 2008 at 7:19 am
JDC325
“Briffa seems to have a habit of making everything absolute rather than relative in order to make a point.”
Well, in this instance, that’s because Lex made an absolute remark. I didn’t make it absolute – it just is. So, I maintain that just one study (the single black swan) would do to refute his argument of ‘hypothesis’ (this is not just good science by the way, it’s simple logic).
So, I think it’s a bit disingenuous of you to claim the relative terms you used make you right, somehow. Because once a hypothesis has been tested, and that hypothesis turns out to be incorrect, then there can’t be any validity in the hypothesis – it’s just wrong.
Where did I crow about getting the scalp of a ‘scientist’ or ‘academic’? Here’s what I wrote:
“And then I wondered if someone with a very pro-science stance would ‘take the bait’, focus on the fact that I only presented one study, and in so doing would betray ‘good science’ (and even logic) and demonstrate the subjectivity and prejudice that is so often found in science and academia.”
As you see, what I referred to was someone with a ‘pro-science stance’. And someone with a pro-science stance does not necessarily need to be a scientist or academic, right? And that’s even if they happen to have an interest in or even work in science of academia (which, by the way, I’m not claiming you do).
And finally, as I mentioned elsewhere, we had a discussion going on my site but now you seem to have somewhat retreated to your own. Please can I encourage you to come back and post your comments on my own site too, so that we may continue our discussions there.
dvnutrix said,
May 31, 2008 at 9:08 am
My goodness, Dr Briffa. You must be a difficult conversationalist if you monitor every conversation according to the rules of semantic football and blow the whistle when you choose to interpret something as absolutist rather than a general conversational style. Lacking your sophistication, I would tend to interpret any remark that uses the phrase “run a mile” in a colloquial fashion rather than affect to believe that it is a linguistic marker inviting a Popperian exchange in the Senior Common Room.
Might I suggest that you attend to the informality of Lex’s style and respond in a more collaborative than combative spirit if you are genuinely interested in discussion rather than highlighting semantic fouls? To the disinterested observer, it merely looks as if you are point-scoring, while claiming the right to make up your rules as you go along.
Becky said,
May 31, 2008 at 9:20 am
Bravo! Never seen anyone tapdance round lexis that creatively. You’re being ridiculous. It would be quite understandable to make the point- “well hold on there they don’t *all* run a mile, let’s not exaggerate [produce one study]” however one would expect you would have the generosity to say ” I take your point of general double blind study avoidance and will address it”. Of if you were feeling particularly snarky to suggest the rewording “did you actually mean this…” In other words yes the absolute hypothesis turned out to be wrong, but only a complete muppet would use that as an excuse not to address the obvious point of his remark. And thankyou jdc325 for posting on your own website so your readers follow this.
John Briffa said,
May 31, 2008 at 9:38 am
dvnutrix and becky
I invoked a basic point of logic and science, but this does not matter so much to you, it appears Would it be fair to say that both of you value style over substance and accuracy with regard to this point?
Call it ‘tapdancing’ or whatever you wish. the fact that Lex was plain wrong (as was JDC325), and your attempt to dress this up as something else is a demonstration of just how little understanding and/or integrity there can be in these sorts of debates.
This does not matter so much here, of course – the fact that Jdc325 made a fundamental error and you are reluctant to admit that is not so much of an issue, really. But this kind of thing can be a very big problem when it comes to things that really do matter, like in healthcare and the safety of vaccinations, which was the focus of my original post was about.
Perhaps you’d like to enter the debate at my own site.
dvnutrix said,
May 31, 2008 at 9:52 am
Oh, I can’t speak for Becky, but I think our meaning is plain but you don’t appear to be someone who is willing to collaborate in communication – a process that typically involves overlooking imprecision in an informal context, such as a blog.
I doubt that many people would agree with your characterisation of yourself or your commenters: it also speaks to your value as a participant in a discussion. Do go and read the Novella piece, you might come away with some interesting food for thought.
John Briffa said,
May 31, 2008 at 10:05 am
dvnutrix
What characterisation of myself are you referring to? And can you explain this: “it also speaks to your value as a participant in a discussion” – I don’t understand what you’re getting at here.
Thanks for linking to the novella piece, it reminded me (not that I needed reminding, to be honest) of how those who claim to be proponents of good science have repeatedly insisted that MMR is safe with regard to autism, when they don’t actually seem to have the evidence that demonstrates that. Talk about ‘changing the rules of evidence’!
Andrew said,
May 31, 2008 at 1:55 pm
The reason, I think, that Briffa is so keen to cast everything in absolutes is because he’s only anywhere near right in that mindlessly pedantic sense in which nobody is really sure that the sun exists.
The MMR vaccine went through all relevant pre-clinical and clinical trials and no significant health risks were found. (I mean “significant” here as in “important enough that the vaccine should not be used”. I don’t mean statistical significance or significance to the person they affect, although I doubt — I’ve not checked this — any of them were significant in those ways either.) Since then the Cochrane Collaboration have done an extensive study of all the research relating to MMR, a total of 139 papers, and found “no credible evidence” for a link with autism.
It would be impossible to conclusively and absolutely prove that there is exactly zero risk, but since there is no credible evidence that this risk exists and since the well-conducted studies looking for exactly this risk have repeatedly failed to find it, it is only true in the the most pedantic, Humean sense that “they don’t actually seem to have the evidence that demonstrates that”.
For any practical considerations, it has been proven that there is no link between MMR and autism. Your denial is a clumsy attempt at an interesting philosophical exercise but from a pragmatic point of view it’s just wrong.
Andrew said,
May 31, 2008 at 2:01 pm
Er, on the subject of things that are wrong, the parenthesis in the second paragraph in that comment wasn’t meant to read quite how it did — it’s entirely likely that a statistically significant risk of insignificantly minor symptoms was found. This would be of no real concern given the huge potential benefits of the jab.
Gosh, did I just admit a mistake? Sorry, I forgot we’re not meant to do that.
John Briffa said,
May 31, 2008 at 2:38 pm
Andrew
On the subject of things that are wrong, please see your latest blogpost and my comments that follow it here:
http://www.apathysketchpad.com/blog/2008/05/30/a-briffas-wrong/
And do please feel free to answer my comments on your site and/or my own.
Also, I posed some specific questions to you on my site but you have not responded. Would you care to do so now?
apgaylard said,
June 1, 2008 at 9:12 am
Fascinating stuff; I’ve been following this (and the other) thread with some interest and would like to make a few comments from a philosophical perspective (sorry for the length of this comment).
It’s always nice to see someone invoking Popper (though the philosophy of science has moved on considerably since – Lakatos springs to mind, and I’m not adverse to a little Khun either); it’s a shame that it was focussed on a side-point: of course Lex overstated the position of homeopaths/nutritionists on evidence. As you pointed out, however, there is some truth in this observation: some well-known apologists for these interventions are trying to run from fair trials, as these don’t give them what they are looking for.
However, this does highlight a key problem with what has come to be known as naïve falisificationalism (a simplistic application of Popper): useful hypotheses can be ‘falsified’ prematurely. It has been observed that all of the great theories of modern physics can be thought of as being born pre-falsified (both Newtonian and relativistic mechanics for example) – evidence against them was already to hand. It took time for the hypotheses to become sufficiently refined, subsidiary hypotheses to be framed, and the evidence against to be scrutinised carefully enough for their value to become clear. Now, I’m not comparing Lex’s ideas with those of Newton or Einstein, but, rather showing that a simplistic Popperian argument isn’t good enough to dismiss them entirely.
Even a careful Popperian (again, not that I’ a Popperian either) may allow the position to be refined to see if there is any value to be extracted from it; rather than prematurely dismissing it. In reality, hypotheses are rarely discarded when they fail a test; they are usually refined or butressed by ancilliary hypotheses.
So rather than worrying too much about white and black swans we need to concentrate on the core of the debate going on here. It is dealing with an important issue: a struggle between two competing ‘programmes’ – one saying that MMR is a good intervention (relatively safe, the benefits outweighing the risks and costs) and the other positing the opposite view. A little philosophy might help this.
Central to this are ideas about ‘proof’ and ‘evidence’.
Proof can mean different things to different people. Some want absolute proof, something like a formal mathematical or logical proof. Unfortunately, that’s the domain of logicians and mathematicians, not the experimental sciences.
Therefore many scientists are, quite understandably, loath to talk about proof. Others do, but they are talking about something more like a commonly understood sort of proof – legal proof. This is where something is considered ‘proven’ by the evidence beyond ‘a reasonable doubt’ by a jury of peers.
Casting this debate in terms of the former is pointless. It’s not what science, or life, is about. Using the latter standard may get us somewhere.
Next, evidence; debates like this usually focus on anomalies – unexpected, controversial results (Wakefield’s imfamous Lancet paper, for example). Advocates for a particular position tend to latch on to them – and go no further. Some even keep hold long after they have been discredited.
From a scientific perspective anomalies are the beginning of a process, not the end. They are rigorously scrutinised, subjected to careful measurement – and mostly evaporate. A real scientific anomaly will survive and help drive change; it emerges from a background of rigorous science and survives intense scrutiny.
Now, for example, it is clear that the MMR/Autism hypothesis of Wakefield did not emerge from a background of rigorous science. Neither has it survived scrutiny. If there are other hypotheses, for which there appears to be evidence, which suggest MMR is not sufficiently safe – I’d be interested to see them debated on this (or the other) thread.
Also, rather than focussing in on individual hypotheses and pieces of evidence a more holistic view is important. We need to bundle together all the hypotheses and evidence associated with each of these ‘research programmes’ and see where they stand relative to each other.
We can then ask: which is more progressive? That is, which is best at addressing perplexing problems; has the most consistency; elegance; simplicity; explanatory power; unifying power; and, dare we say, truth about it? (see here for a description of this more relevant, contemporary Lakatosian, approach.)
I’d say that the answer is pretty clear. All things considered the ‘MMR is a valuable intervention’ programme is progressive. What swings it for me is the lack of rigour in the oft-cited anomalies and the direction the best evidence points in. It’s not perfect. Science, like life, does not come with absolute guarantees. But it can point us in the most useful, helpful and productive direction.
The stuff of science is not sterile (and synthetic) debate about proving absolutely that an intervention can never harm; but rather settling reasonable doubt by looking at the balance of carefully tested evidence.
jdc325 said,
June 2, 2008 at 12:34 pm
You claimed to demonstrate the subjectivity and prejudice that is so often found in science and academia – I am not involved in either. Have you really demonstrated “the subjectivity and prejudice that is so often found in science and academia”?
DT said,
June 5, 2008 at 11:52 pm
Dr Briffa, do you care to comment on apgaylard’s quite devastating exposition of scientific philosophy as it applies to MMR and how it exposes you as wearing the emperor’s new clothes?
I thought not…….
Well then, how about your views on the numerous studies quoted in the IOM report which provide ressurance as to the safety of MMR in regard to autism?
A critical rebuttal of each paper, in turn, would do.
I won’t hold my breath. I can see you prefer to regard an opinion piece from a lawyer who makes considerable sums in vaccination claim cases as “scientific evidence”, despite the barndoor conflict of interest that accompanies his every utterance (every time he slags off MMR he stands a greater chance of success in court). Tell us, why do you hold “evidence” of this nature in such high esteem. Recall, Clifford Miller is a man who claims that scientists’ experimental observations count as “anecdote”, and that court witness testimony qualifies as “scientific evidence”.
Living with uncertainty « A canna’ change the laws of physics said,
June 11, 2008 at 9:29 pm
[…] been following a debate between various bloggers and Dr John Briffa about MMR and autism. The good doctor seems to take […]
AltMed Support for Wakefield Continues « jdc325’s Weblog said,
March 15, 2009 at 7:53 pm
[…] some MMR scaremongering of his own (Briffa’s original post), which I covered here, here, and here [note: the first of my blog posts includes links to many other blogs covering Briffa’s […]