Right – I’ve probably got this completely wrong.*
Following yesterday’s whinge about not being able to read a study that the Beeb had reported on, some kind soul has emailed me the PDFs of the studies that the BBC seem to be basing their news report on. I’m really not qualified** to comment on the TCHM papers, but here are some observations/questions anyway:
The TCHM group had higher baseline score for number of days per month when Corticosteroids were used.
Patients in the TCHM group went from 19 days per month in the first month to 16 days in the second and third months and 15 days in the fourth month.
Patients in the placebo group went from 17 days in the first month to 16 days in each subsequent month.
Could it be that the difference in CS use stems from the fact that the TCHM group was worse off to begin with? Is this an example of regression to the mean?
TCHM vs placebo:
Upper respiratory tract Infection 8-9
Abdominal pain 4-2
Episodes of asthma 6-3
New rash 8-5
GP visits 21-11
Antibiotic for > 3 days 12-6
Topical ⁄ oral corticosteroid 5-1
There were more adverse events in the TCHM group for each category (with the exceptions of upper respiratory tract infections and hospitalisation). Other medical treatments were required more often in the TCHM group than the placebo group. Funnily enough, the BBC didn’t mention the adverse reactions in the TCHM group.
Note: This study was conducted in the People’s Republic of China. This study – Vickers A, Goyal N, Harland R, Rees R. Do certain countries produce only positive results? A systematic review of controlled trials. Controlled Clin Trials 1998; 19: 159-166 – states that “some countries publish unusually high proportions of positive results. Publication bias is a possible explanation. Researchers undertaking systematic reviews should consider carefully how to manage data from these countries”, that in their review of acupuncture studies “all trials originating in China, Japan, Hong Kong, and Taiwan were positive, as were 10 out of 11 of those published in Russia/USSR” and that in their review of trials of treatments other than acupuncture “no trial published in China or Russia/USSR found a test treatment to be ineffective.” Abstract here.
The paper involving 85 patients was accepted in January 2007. The in vitro paper was submitted in December 2007. The Beeb reported on these findings as if there were just the one paper. I don’t know about anyone else, but I found that slightly confusing and I’m not sure why they did it.
The BBC managed to find room for all the positive findings of the studies, but must have run out of space when it came to the adverse events, as these were not reported on the BBC website. Why publicise all the positive findings and none of the negative findings?
The Beeb wrote that “further analysis revealed that patients who took the herbal remedy showed lower blood levels of four proteins thought to have inflammatory effects linked with eczema” – but it isn’t clear where this further analysis can be found. They also wrote that “this finding was confirmed in lab tests in which the pentaherbs formulation was added to blood cells in a test tube” – the lab tests they refer to here appear to be the basis of the second paper (in vitro tests on peripheral blood mononuclear cells). This second paper refers to a third, “effects of a 3-month, open-label study of PHF on circulating inflammatory mediators in children with AD” – which could be the ‘further analysis’ referred to earlier. The study is this one: Hon KL, Leung TF, Wong Y et al. A pentaherbs capsule as a treatment option for atopic dermatitis in children: an open-labeled case series. Am J Chin Med 2004; 32:941–50
So how many papers are the BBC reporting on here? One, two or three? It would have been so much easier to find out about these studies if the BBC had referenced them. I think I might just ignore unreferenced news reports on scientific studies in future. Apart from cobbling together a story using bits-and-pieces from three different papers (none of which were referenced!), the BBC’s opening paragraph stated “A traditional Chinese herbal medicine consisting of five herbs may ease eczema symptoms, a study suggests.” – I don’t think this is a fair reflection. They claim that the TCHM may ease eczema symptoms, but the studies claimed to show: “improved quality of life”; reduced use of CS; reduction of markers of inflammation (in vitro). Not a decrease in symptoms.
*This is my new motto. Assume this to be true for anything I post.
**I should point out that this isn’t false modesty – I’m really not qualified at all. I’m not even a scientist. I’m not going to let that stop me from bitching about the way science is reported in the media though.
***Thanks are due to Jellytussle and BadChemist.