I can read it!

March 14, 2008 at 1:43 pm (Bad Science, Media, Remedies) (, , , , , , )

Right – I’ve probably got this completely wrong.*

Following yesterday’s whinge about not being able to read a study that the Beeb had reported on, some kind soul has emailed me the PDFs of the studies that the BBC seem to be basing their news report on. I’m really not qualified** to comment on the TCHM papers, but here are some observations/questions anyway:

Corticosteroid Use:
The TCHM group had higher baseline score for number of days per month when Corticosteroids were used.
Patients in the TCHM group went from 19 days per month in the first month to 16 days in the second and third months and 15 days in the fourth month.
Patients in the placebo group went from 17 days in the first month to 16 days in each subsequent month.
Could it be that the difference in CS use stems from the fact that the TCHM group was worse off to begin with? Is this an example of regression to the mean?

Adverse Events:
TCHM vs placebo:
Upper respiratory tract Infection 8-9
Diarrhoea 3-0
Abdominal pain 4-2
Episodes of asthma 6-3
New rash 8-5
Miscellaneous 6-4
Hospitalization 2-2
GP visits 21-11
Antibiotic for > 3 days 12-6
Topical ⁄ oral corticosteroid 5-1
There were more adverse events in the TCHM group for each category (with the exceptions of upper respiratory tract infections and hospitalisation). Other medical treatments were required more often in the TCHM group than the placebo group. Funnily enough, the BBC didn’t mention the adverse reactions in the TCHM group.

Note: This study was conducted in the People’s Republic of China. This study – Vickers A, Goyal N, Harland R, Rees R. Do certain countries produce only positive results? A systematic review of controlled trials. Controlled Clin Trials 1998; 19: 159-166 – states that “some countries publish unusually high proportions of positive results. Publication bias is a possible explanation. Researchers undertaking systematic reviews should consider carefully how to manage data from these countries”, that in their review of acupuncture studies “all trials originating in China, Japan, Hong Kong, and Taiwan were positive, as were 10 out of 11 of those published in Russia/USSR” and that in their review of trials of treatments other than acupuncture “no trial published in China or Russia/USSR found a test treatment to be ineffective.” Abstract here.

BBC reporting:
The paper involving 85 patients was accepted in January 2007. The in vitro paper was submitted in December 2007. The Beeb reported on these findings as if there were just the one paper. I don’t know about anyone else, but I found that slightly confusing and I’m not sure why they did it.
The BBC managed to find room for all the positive findings of the studies, but must have run out of space when it came to the adverse events, as these were not reported on the BBC website. Why publicise all the positive findings and none of the negative findings?
The Beeb wrote that “further analysis revealed that patients who took the herbal remedy showed lower blood levels of four proteins thought to have inflammatory effects linked with eczema” – but it isn’t clear where this further analysis can be found. They also wrote that “this finding was confirmed in lab tests in which the pentaherbs formulation was added to blood cells in a test tube” – the lab tests they refer to here appear to be the basis of the second paper (in vitro tests on peripheral blood mononuclear cells). This second paper refers to a third, “effects of a 3-month, open-label study of PHF on circulating inflammatory mediators in children with AD” – which could be the ‘further analysis’ referred to earlier. The study is this one: Hon KL, Leung TF, Wong Y et al. A pentaherbs capsule as a treatment option for atopic dermatitis in children: an open-labeled case series. Am J Chin Med 2004; 32:941–50
So how many papers are the BBC reporting on here? One, two or three? It would have been so much easier to find out about these studies if the BBC had referenced them. I think I might just ignore unreferenced news reports on scientific studies in future. Apart from cobbling together a story using bits-and-pieces from three different papers (none of which were referenced!), the BBC’s opening paragraph stated “A traditional Chinese herbal medicine consisting of five herbs may ease eczema symptoms, a study suggests.” – I don’t think this is a fair reflection. They claim that the TCHM may ease eczema symptoms, but the studies claimed to show: “improved quality of life”; reduced use of CS; reduction of markers of inflammation (in vitro). Not a decrease in symptoms.

*This is my new motto. Assume this to be true for anything I post.
**I should point out that this isn’t false modesty – I’m really not qualified at all. I’m not even a scientist. I’m not going to let that stop me from bitching about the way science is reported in the media though.
***Thanks are due to Jellytussle and BadChemist.

3 Comments

  1. coracle said,

    I remember seeing the article on the bbc and I’m glad someone has looked into it. So the authors are claiming a significant difference between 16 days/month CS use with the herbs and 17 days/month on placebo, is that right?

    Unless the standard deviations are really tight, I’d have difficulty believing a significant difference. Alternatively, having looked at the discussion on this on the badscience forums, are the authors claiming a significant difference in the time taked to achieve this reduction in CS use?

    Either way, I’d wonder whether there was a clinical significance in using these herbs, a single day less CS use per month hardly seems to justify it.

  2. coracle said,

    Just to update myself on this, the excellent Behind the headlines have a good article on this. Makes a lot of things clear.

  3. jdc said,

    Thanks for the link and comments Coracle.

    I actually think that the most annoying thing (for me) about the BBC coverage is that their report made it so difficult to get hold of the original research. They used three papers and failed to cite a single one. They then reported on all the positive results of the papers (improved quality of life, reduced use of steroids) and ignored the negative (e.g., more adverse events in the TCHM group)

    One other thing the BBC didn’t report was that over 12 weeks, the mean SCORAD score fell from 58.3 to 49.7 in the TCHM group and from 56.9 to 46.9 in the placebo group. The TCHM group had an average reduction of 14.75% and the placebo group had an average reduction of 17.57%. The BBC could quite easily have reported this, along with the adverse events data, and concluded that TCHM was inferior to placebo and produced more side-effects. I suppose my point is this: if they’re going to be selective about what data they use and they aren’t going to cite the papers they got the data from, they are effectively providing PR rather than news.

    This (from http://www.badscience.net/?p=630) is probably relevant to my post as well: “Big pharma is worth £150bn, it pushes products to doctors, who are trained to spot drug company nonsense. The food supplement industry is worth £30bn, and is marketed at you, with sciencey-sounding rhetoric, through the media, where no regulator will touch their claims.” If the BBC are reporting on OTC herbal products, food supplements etc then they are reporting on things that readers/viewers can go out and purchase for themselves (i.e., without any input from a medical professional who would be best placed to advise them) and so they really should provide full, accurate and unbiased information. Otherwise, they end result is that they are just pushing dubious remedies that are sold direct to the public.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: