I’ve written previously on suposed ‘trials’ of the benefits of fish oil on kids’ school performance. One was the ‘Durham trial’, where a large cohort of schoolchildren was given fish oil supplements without any real scientific basis for doing so; another was in New Zealand. As I said then, one of the many problems with this ‘trial’ was the absence of any proper control treatment. The Bad Science website written by Ben Goldacre had a very thorough analysis at the time.
Now Ben’s just written about a subsequent trial which was properly controlled. I wasn’t particularly surprised to hear that it showed no difference in outcome between ‘oil’ & ‘control’ groups. Unless – & this is actually the focus of Ben’s post – you do a subgroup analysis. This is essentially where you cherry-pick a subgroup of the data set, because the members of that subgroup do appear to show a response to the treatment. And it’s a Bad Thing, really. Ben explains why.