Chiropractic for Colic: A Planned RCT

August 13, 2009 at 2:16 pm (Alternative Medicine, Chiropractic) (, , )

According to this, a randomised double-blind placebo-controlled trial into the effects of chiropractic treatment on infantile colic is to take place. Some will say “too little too late”, given that chiropractors have been claiming to “treat” or “help with” colic for some time now. I have a different concern. The notes on the planned trial state that “subgroup analyses will be performed to identify possible subgroups of infants who will benefit the most from the treatment”. My concern is that, even in the event that the trial does not show a benefit for chiropractic treatment of colic, advocates may claim that subgroup analysis shows that there is a benefit for some children and therefore chiropractic works as a treatment for colic. [The remainder of this post will consist mainly of copying and pasting from here and one or two other sources.]

Ben Goldacre wrote in April about a trial of fish oil pills. Here are some quotes from the piece that I found interesting:

The fish oil pills did nothing, as in many previous studies, so this trial has not been press released by the company, nor has it been covered in the media. The funders of this study, Equazen, will doubtless have been disappointed with a negative result. The authors of the study may have been disappointed too. But there was some light on the horizon. They looked at the data more closely, and found that some children did, in fact, respond: “a subgroup of 26% responded with more than 25% reduction of ADHD symptoms and a drop of CGI scores to the near-normal range.”

Interestingly it turns out that you can show significant benefits, using a subgroup analysis, even in a fake trial, where the intervention consists of doing absolutely nothing whatsoever. 30 years ago Lee et al published the classic cautionary paper on this topic in the journal Circulation* […] They were not disappointed. Overall, as expected, there was no difference in survival between the two groups. But in a subgroup of 397 patients (characterized by “three-vessel disease” and “abnormal left ventricular contraction”) the survival of Treatment 1 patients was significantly different from that of Treatment 2 patients. This was entirely by chance.

You can also find spurious subgroup effects in real trials, if you do an analysis that’s foolish enough. Close analysis of the ECST trial found that the efficacy of a procedure called endarterectomy depended on which day of the week you were born on. Base your clinical decisions on that: I dare you.

At one point in the article, Goldacre points out that:

Subgroup analyses are widely derided in academia, and for very good reasons.

This article in the NEJM, meanwhile, begins by telling us that:

Subgroup analyses are an important part of the analysis of a comparative clinical trial. However, they are commonly overinterpreted and can lead to further research that is misguided or, worse, to suboptimal patient care.

I look forward to reading the trial of chiropractic for colic and hope that the subgroup analyses will not be overinterpreted and lead to misguided research or suboptimal patient care.

More

*This [PDF] is the Circulation paper that is mentioned in the Bad Science article.

h/t Zeno

4 Comments

  1. Zeno said,

    As you say, it’s a bit late to try to come up with a justification for what’s been claimed for years.

    And I take your point about the sub-groups: if the main results don’t tell you what you want to hear, keep digging…

  2. Michael said,

    A minor correction: the planned trial is single-blind, as only the parents will be blinded. Studies of the placebo effect show that it can make a difference if trial investigators are not blinded. So the study starts off with a methodological weakness.

    The protocol hints that subgroups will be infants with and without suspected musculoskeletal problems. But, it does not say what the musculoskeletal problems are, or how they will be diagnosed or suspected. If they choose the infants who are crying the most, they will be using the “regression of the mean” effect to bias the results in favour of chiropractic.

  3. jdc325 said,

    Thank you Michael. The Clinical Trials page still states that it will be a double-blind trial, so I might try to contact them for clarification of this point. Thank you also for the comments regarding placebo and regression to the mean.

  4. Liz said,

    I worry about this too, chiro is one of the only treatments I’ve never had a mom mention to me as working, and it DID NOT work for my babies either.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: