Recently I was asked by a scientific journal to review an article that had been sent to them. This is pretty standard procedure for journals, and every scientist will know what I am talking about. For those non-scientists, peer review is a way of ensuring (or rather, trying to ensure) quality in scientific publication. If someone writes an article and submits it to a scientific journal, the editors will send it to people reasonable competent in the field for them to comment on. For example, is the subject matter appropriate for the journal? is the method sound? are the results believable? are the conclusions backed up by the results? etc etc.
But peer review at the journal stage is not fool proof. There have been cases where downright untrue articles have made it into print (and in top class journals too). I’ve already discussed one case on this blog. And it’s not surprising, since there is a lot of reliance on honesty in the process. As a reviewer, I will not (usually) go and repeat the work that the author is describing to check that he’s telling the truth – this would likely take too long (reviewers don’t get paid for their role), require specialist apparatus that I don’t have available to me at the university, or be too costly. What I will do, amongst other things, is check the suitability of someone’s method, the plausibility of his results, and the validity of his conclusions. If someone deliberately sets out to mislead over their results, it may be a long time before it’s picked up.
There is often the perception that if an article has got through a journal’s peer-review and is published, that means it is right. Not true. What it means is that it’s suitable for the wider scientific community to look at. Some work is found to contain mistakes – perhaps in methodology, perhaps in processing of data, perhaps in invalid assumptions – and this is all part of the wider peer review that it hits when it makes it into print. These mistakes are not necessary malicious – they can be honestly made – and a good scientist would welcome them being picked up in his work (In fact, he probably would be flattered that someone is reading his work very carefully.)
Journal peer review isn’t a fool-proof method, but is probably about as good as we can realistically expect to get. Winston Churchill commented: