Sometimes I suspect that you think I’m over-reacting to poor reporting of science stories in the media. Maybe I’m just picky, or pernickety, or – as my Significant Other would says – purely pedantic. So I was interested to see a paper in PLoS One (Ly & Lane, 2009) that looks at the quality of various medical research stories that make it to newspaper front pages. It’s interesting stuff:
The researchers searched a news database to find medical research stories that made it to the front pages of major newspapers during the period January 1, 2000 to December 31, 2002. This yielded a total of 734 front-page articles. They then used 2 search engines, Medline & Google Scholar, to find the corresponding journal articles, & decided whether the research was ‘mature’ (ie with a solid research basis, published in the peer-reviewed literature) or ‘preliminary’ (an interesting finding but one that needed further research work to support it, & presented at a conference or a press briefing).
It turns out that only 57% (417) of those newspaper stories had been published in peer-reviewed journals (and they tended to have a higher evidence level than the remainder). The rest were picked up by the journalists via conferences or press briefings, although 144 of those subsequently reached the stage of being published in the scientific literature. So that leaves 173 pieces of research that never went any further but – unfortunately – still received the same weight in the press & in the public mind as those items with a stronger evidence base behind them.
Yes, not one of those 734 news stories indicated the level of evidence behind the story, and less than 20% of them made it clear that the news article was reporting on preliminary findings. But at least some of the peer-reviewed work, and many of the preliminary outcomes, would later have been rejected on the basis of further research – because that’s how science research pans out. Much of what we do turns out to be ‘wrong’ in the sense that we don’t get the answers that our initial hypotheses predict. Nothing wrong with that at all, but that side of things rarely makes the headlines. ‘X is a cure for cancer’ is going to sell papers, in a way that ‘X is not a cure for cancer’ most definitely will not.
And as a PS – here is an excellent example from Ben Goldacre about how badly a news report can get it wrong.
W.Y.Y.Lai & T.Lane (2009) Characteristics of Medical Research News Reported on Front Pages of Newspapers. PLoS ONE 4(7): e6103. doi:10.1371/journal.pone.0006103