Hypothesis: I reckon there are loads of claims made in the press that don’t have adequate (or sometimes any) evidence to support them. As an example, I took a quick look at the mentions of retinol and wrinkles in UK publications over the last two years.
Method: I used a database search to extract articles that referred to retinol and wrinkles in the same paragraph. I specified a period of two years, and also specified UK publications. Once I’d grabbed the articles, I noted the ones that made claims for retinol in relation to its supposed anti-wrinkle properties. I then looked at how often publications featured, how many made reference to scientific studies supporting the claim and which studies were cited most often. I also looked at how many gave a proper citation and how many simply referred to the study by naming the journal in which it appeared or the place the study was conducted.
Results: Of 47 results, seven were duplicated. Of the remaining forty, not a single one gave a full citation. Ten referred to a study supporting the claims made, thirty made no reference to a scientific study. Of the ten, one referred to a study and gave no detail whatsoever – making it impossible even to guess at the source of their information – and one referred to a study conducted by Olay and gave no further detail. A further eight articles referred to a study in the Archives of Dermatology conducted at the University of Michigan. Seven of the articles referred to pro-retinol and thirty three to retinol. The Chemist and Druggist featured twice, Coventry Evening Telegraph once, the Daily Mail five times, Daily Record twice, while the Daily Star, Daily Telegraph, Evening Gazette, and Evening News each featured once. The Express appeared four times, the Metro once, the Mirror three times and the News of The World and Northern Echo once each. The Observer was in there twice, the Sunday Times and the Sun thrice, while the Times made it five, um, times. Pulse, Source, The Guardian, Western Daily Press were all singletons. The only articles that did not make claims for retinol or pro-retinol were one of the five articles in the Daily Mail and the sole article that appeared in Pulse magazine.
Waffle: While forty newspaper and magazine articles mentioned retinol and wrinkles in the same paragraph over the last two years, the only references to scientific evidence for retinol seemed to be to the University of Michigan study (that wasn’t actually cited in the articles, but was referred to in order to lend weight to the claims being made). This study was published in the Archives of Dermatology [here]. There were 36 subjects, with an average age of 87 years. “Topical 0.4% retinol lotion or its vehicle was applied at each visit by study personnel to either the right or the left arm, up to 3 times a week for 24 weeks.” Now, quite apart from the fact that some of the creams that were being written about by the newspapers and magazines contain pro-vitamin A (i.e., retinoids or beta carotene – which is a bit different from them containing actual retinol), this seems an awfully small study on which to base claims about the effectiveness of a cosmetic cream and I’m not sure that effects in 87-year-old subjects will be generalisable to the population that makes up the target group for advertisers of cosmetic creams. As it turns out, 13 participants dropped out of the study, so we are actually talking about a study containing 23 subjects that is being used to help market retinol-containing “anti-ageing” creams that will, according to manufacturers and retailers [and the press!], reduce visible wrinkles.
Limitations: Loads, probably – and most of them won’t even have occurred to me yet. Here’s some examples though: The publications didn’t include women’s magazines [which I would expect to be a good source of anti-ageing cream claims]. I didn’t come up with a proper checklist prior to going through the articles, I just looked to see whether they had made claims for some type of retinol in relation to its anti-ageing or anti-wrinkle properties and whether they had referred to any kind of scientific evidence. I didn’t bother looking at the articles that had made unevidenced claims and ask them [the authors of the unreferenced articles] where, precisely, they had got their information from. I gave absolute figures for the number of mentions without accounting for which of the papers and magazines were daily, weekly or monthly. I only looked at two year’s worth of articles which, once I’d removed duplicates, came to a grand total of forty – too small a sample really.
This is for you Nash: retinol [PDF]