how not to do science: the scole experiment

I listen to quite a lot of podcasts. Lately I’ve been listening to more than usual. I’ve had the flu (I’m assuming that’s what it was, since colds tend not to come with fever, chills, & sore joints) & listening to stuff was easier than reading. Anyway, I digress.

One of my current favourite podcasts is Brian Dunning’s Skeptoid.com. – excellent primers in critical thinking, nicely presented, & not too long. One of these concerned the ‘Scole experiment’ – supposedly an excercise in which scientists tested the claims of mediums (people claiming to communicate with the spirit world) and  – gasp! – found the claims justified. I’ve been interested in claims about the paranormal ever since reading (& re-reading, multiple times) Martin Gardner’s book Science: good, bad & bogus.

The Scole experiment – named for the village in England where it was carried out – was a series of seances led by 6 mediums and investigated by 15 members of the Society for Psychical Research. Involving large numbers of investigators & psychics, It supposedly provided evidence of the existence of ‘spirits’: lights moving about in a darkened, closed room; photographic images appearing on film that had previously been sealed into secure containers, physical contact with invisible entities; tables lifting off the ground, & voices coming out of nowhere. Nor, it’s claimed, was there any evidence of fraud.

So, evidence that there is an afterlife, & us sceptical sorts should start to revise our worldview? Not so fast, says Dunning.  When you come to look closely, the Scole experiment turns out to be a very good example of how not to do a scientific investigation.

I know that if I was going to undertake an investigation of psychic claims, about the first thing on my list would be to put in place various controls & restrictions, thus minimising the opportunity for any possibility of fraud. Things like cameras, motion sensors, venues thoroughly checked beforehand. Amazingly, Dunning tells us that this was not what happened in the Scole study. Here, the mediums set all the rules, thus effectively ruling out the possibility of this being a serious scientific examination of the proceedings. In fact, it appears that the investigators did everything that the mediums asked of them – Dunning describes them as acting as an audience, rather than researchers. Thus:

• the psychics were effectively free to move around during each seance (no hand-holding), and thus the investigators weren’t able to exclude the possibility of the phenomena they witnessed being generated by the performers themselves.
• they banned any use of still or video cameras – & this included infra-red & night-vision equipment. (One has to wonder why this was the case – if the seance was genuine, this technology would surely be no threat..)
• the box into which unexposed films were locked was supplied, not by the investigators, but by the psychics. What’s more, one of the investigators wrote that he was able to easily open the box in the dark… Films placed in boxes supplied by the ‘researchers’ never developed any images, a fairly suggestive finding.
• the seances were carried out in a room provided by the mediums, not the researchers.
• and – despite the fact that the Scole performances have been hailed as proof positive of an afterlife – there’s been no follow-up at all.

This complete lack of any serious controls and experimental protocols means that we can’t take the study’s findings seriously. And it does bring to mind a couple of comments from Gardner’s book. One has to do with the apparent hypersensitivity of psychic phenomena to any sort of critical examination. The other was Gardner’s statement (I think originally made by prestidigitator & debunker James Randi) that scientists are perhaps the easiest audience to fool  if you’re a magician or psychic – because they don’t expect anyone to be setting out to deliberately pull the wool over their eyes…

I do enjoy my regular Skeptoid fixes 🙂