After I posted Food for thought?, I got a message from a student saying that she'd seen the study reported on the Documentary Channel. She thought the results looked good, but commented
… it never stated if they had a control, possibly placebos, so that it can be assesed whether some of the children merely concentrated more because they believe they should, or if this experiment was actually achieving such positive results.
And that's spot-on. For data from the Durham trial to be scientifically valid, you might expect that there'd be a control group of students who didn't receive the fish oil capsules. Of course, if this happened then the students would know whether or not they were getting the oil, & – as my correspondent notes – this could affect their behaviour.
A better trial design would involve a placebo: split the students into two groups, matched for age, gender, perhaps ethnic group & social class. Give one group fish oil capsules, and the other capsules which look and taste the same but don't contain any fish oils. And double-blind the study, so that neither the students nor the researchers know which group a student belongs to. It's hard to tell if this was done for the Durham trial; the only paper I can find refers to an earlier study on a much smaller group of students (N=117, all with 'developmental coordination disorder).
Did the oil treatment have any effect on the students' exam results? The cohort of students from Durham who sat GCSE exams in the previous year (no fishy treatment) improved 5.5% on the cohort before them. The fish-oil cohort improved 3.5% on that (the figure across the UK was 2%). How would you interpret those figures?