the carnegie hall hypothesis: practice makes perfect

Hot on the heels of the paper on methods for improving learning in first-year physics (Deslauriers, Schelew & Wieman, 2011), comes one by Haak, HilleRisLambers, Pitre & Freeman (2011) that casts a critical eye on methods for teaching first-year biology classes.

Today’s students come from more diverse backgrounds, and have far more diverse prior learning experiences, than when I was a student myself. Those differences can contribute to a gap in achievement in first-year biology – something that’s exacerbated by academic assumptions about prior learning & which can contribute to poor student retention into subsequent study in the subject. 

Interestingly, when I showed the paper to a couple of colleagues, their first response was ‘let’s get clickers!’. Personally I’m rather cool on the idea: partly because they’re not exactly cheap, but also because what both papers show is that it’s not the clickers themselves that make the difference, it’s what you do with them. If all you do is use them to find out what answers students give to multichoice questions, & nothing more, this technology won’t actually add anything to student learning (eg Deslauriers et al., 2011). If, however, clickers are used as an integral part of a wider, active learning, experience, then you’ll see biiig improvements in student learning outcomes.

And that is amply demonstrated by this latest study by Haak & his colleagues. They note that, especially when dealing with disadvantaged students in the US, the response has often been to throw money at the problem to support reasonably comprehensive heavily targeted programs. Because this can quickly become rather expensive, such programs rarely become a regular, normal part of teaching programs. And they ask:

Can an existing STEM course [1] be modified to improve performance by students from disadvantaged educational and socioeconomic backgrounds who are at high risk of failing, without requiring increased resources in the way of staffing or external funding?

In other words, is it possible to set things up in a large-group lecture classroom that lets students achieve as they would if they were getting one-on-one instruction? This is something that should be of interest to university academics for several reasons, including the fact that in New Zealand there is an increasing focus from the government on improving student retention and completion rates, and on enhancing the proportion of Maori & Pasifika students enrolling & succeeding in university programs.

To answer this question, the research team worked with a big first-year Biology class at the University of Washington, specifically looking at the performance of students in the university’s Educational Opportunity Program (EOP). Students in this program come from disadvantaged backgrounds (educationally &/or economically), & the majority of them are from non-Caucasian ethnic groups. Analysis of a very large number of student records found a large ‘achievement gap’ between EOP and non-EOP students, such that over the period 2003-2008 EOP students in Biology 180 had an average failure rate of 21.9% (cf 10.1% for the non-EOP cohort). Haak & his colleagues hypothesised that this was because grades in the paper were heavily dependent on exams that ‘test higher-order cognitive skills’, and that students in the EOP program aren’t as well prepared as the non-EOP group to that assessment style.

The paper was originally taught by the standard, traditional lecture format, with little involvement by the students. Previous work led by one of the team (Freeman) found that if the lecturer incorporated active-learning exercises in his class (daily multichoice questions and weekly practice tests), then all students’ performance improved compared to the outcome from that traditional format. This makes sense, as the students were practicing the skills they’d need to achieve well in the final exam.

But wait, there’s more. A third course design saw the class taught (by the same instructor) without any lectures at all, where the active-learning exercises were combined with ‘pre-class reading quizzes and extensive informal group work in class’ – exactly what Deslauriers & his team did with their experimental cohort of physics students.  No surprises here:

The highly structured [third] approach resulted in another increase in overall performance by all students, compared with the low-structure, lecture-intensive course with no required active learning and [my emphasis] the moderate structure design based on clickers and a weekly practice exam.

That in itself is an excellent outcome. What about the EOP students in particular, since that’s where the big achievement gap is apparent?

… although all students benefit from [highly-structured teaching], EOP students experience a disproportionate benefit.

Way to go! Importantly, in these straitened economic times, this intervention didn’t cost any extra money. What’s more, the second time the ‘highly structured’ intervention was used, class size had gone from 345 to 700, lab clases had been cut to one every 2 weeks, and the ratio of teaching assistants to students went from 1:49 to 1:87.5. (Note to the Finance people: this is not a reason to cut funding for demonstrators!)

You could ask how, exactly, this intervention is having its effect. Are the students simply learning more ‘stuff’ as a result of the different teaching methods, or are they also gaining higher-order cognitive skills? During Cathy Buntting’s PhD research she found that teaching students how to develop concept maps had a significant impact on their ability to answer ‘thinking’ questions, as opposed to ‘recall’ questions, so I’d have put money on Haak’s team finding that active learning has a positive impact on cognitive abilities. Haak & his colleagues comment that because Biology 180 relies heavily on higher-order thinking-type questions in its exams, then better results in those exams does suggest ‘actual learning gains’ and an improved understanding of the content covered in the paper. They suggest that

active learning that promotes peer interaction makes students articulate their logic and consider other points of view when solving problems, leading to learning gains.

Hopefully this will be the focus of a future research project.

Deslauriers L, Schelew E, & Wieman C (2011). Improved learning in a large-enrollment physics class. Science (New York, N.Y.), 332 (6031), 862-4 PMID: 21566198

Haak DC, HilleRisLambers J, Pitre E, & Freeman S (2011). Increased structure and active learning reduce the achievement gap in introductory biology. Science (New York, N.Y.), 332 (6034), 1213-6 PMID: 21636776

[1] STEM = Science, Technology, Engineering & Mathematics

Oh, and the ‘Carnegie Hall’ hypothesis? It’s named for the story of a tourist who asked a New Yorker how to get to Carnegie Hall. The local guy answered, ‘practice!’

6 thoughts on “the carnegie hall hypothesis: practice makes perfect”

  • Jim Thomerson says:

    A student who had the lowest test scores I have heard of took my straight lecture general education biology class. She failed to the max. One of my faculty used the same class the next semester to develop an implement a program of “Student Assisted Learning”. Basically group sessions with an upper class student guiding the discussions were available on a regular basis. The failed student took advantage and attended every scheduled discussion session. She made a solid middle B in the course. Although the statistics could be questioned, students who attended some of the sessions generally improved their performance.

  • Alison Campbell says:

    Great to hear that your colleague’s program was so helpful for your student 🙂 There’s a lot of evidence that such interventions are effective; the trick is always going to be to get admin to pay for the classes & students to attend them. That’s why I think the changes in pedagogy that both Deslauriers & Haak describe are so effective: admin like them because they work & don’t cost any more, & students have to go. Well, they still have the option, but the evidence does show that they’re more likely to opt in to active learning sessions than to opt out.

  • I’ve said it before- we are falling through the cracks. So many of my peers (myself included) do not respond well to lecture style learning. I am so glad to read that this dealt specifically with EOP students.
    I came from a far-from-wealthy family and a broken public school system. I wasn’t prepared for college, and three years in, I’m still playing catch up.
    These are real problems that need to be addressed. I’m not the smartest kid in the class, but there is a serious issue when so many people like me are being left behind in the system.
    Great blog, btw!

  • Alison Campbell says:

    Thanks for writing, Mike. I do so hope that you manage to work your way through this & come out successfully at the other side.
    As you’ll have seen, this is something I feel very strongly about. The ‘system’ is difficult to change but we do seem to be making small steps here – I’ve got quite a few colleagues around this university who are innovative educators & don’t follow ‘the lecture’ format. And around NZ too – we’ve got an awards system for tertiary educators who are outstanding when it comes to maximising student learning success (by whatever means).

  • At the University of Auckland I introduced the use of several engagement strategies and improved the pass rate from 83% to 93% with half the cohort achieving A grades. In lectures I used clickers (though only 1/3 students participated), and also group problem solving at least once each lecture in the series of 12. In tutorials we used both qualitative and quantitative problems, and in problem solving laboratories we used Team Based Learning (involving pre-session reading, then both individual and team scores. TBL is very well researched and very successful!).

Leave a Reply

Your email address will not be published. Required fields are marked *