Reading Comprehension Studies Can Mislead Us About What Works

Nathaniel Hansford, a Canadian teacher and education blogger, is the lead author of a newly published meta-analysis of 71 studies of reading comprehension. It’s the result, he writes, of three and a half years spent screening over 1,500 studies and re-running analyses countless times—which involved “learning far more statistics than [he] ever planned to.” As far as I can tell, Hansford doesn’t have a formal background in statistics or mathematics, so this is an impressive achievement.

But the meta-analysis fails to provide much useful guidance. Do the benefits of comprehension strategy instruction plateau after just a few hours, as some have argued? Maybe yes, maybe no. Ditto, at least according to Hansford, for whether the benefits of building students’ knowledge increase the longer the process continues.

In any event, given that the knowledge-building interventions in the studies didn’t involve coherent knowledge-building curricula—they just incorporated some science or social studies content into literacy instruction—Hansford cautions that “conclusions drawn here should not be interpreted as direct evidence for or against knowledge-building models.”

Still, the meta-analysis contains a strong undercurrent of skepticism about the benefits of knowledge-building for reading comprehension, and it’s something other commentators have picked up on.

On his Substack, David Didau has described Hansford’s finding of relatively strong evidence for reciprocal teaching—which incorporates four comprehension strategies—as “inconvenient for advocates of ‘knowledge-rich’ approaches to education.” Harriet Janetos, another education Substacker, took the release of the meta-analysis as an occasion to criticize the “knowledge-building movement” for “setting up No Strategies Allowed signs” rather than identifying the strategies that are most likely to be helpful.

Speaking as an advocate of knowledge-building, I don’t consider the findings in Hansford’s meta-analysis “inconvenient.” Rather, I find the whole endeavor of questionable value, partly because of the faulty premises of the underlying studies and partly because I believe the meta-analysis understates the evidence for knowledge-building.

I also don’t think it’s fair to say that the knowledge-building movement has set up “No Strategies Allowed” signs. Of course, there’s no official leader of the movement and consequently no single disciplined message. Some knowledge advocates may have argued, or appeared to argue, that all strategy instruction should be verboten.

In any event, it’s understandable that knowledge advocates have targeted skill-and-strategy instruction, given that it routinely displaces subjects like social studies and science from the curriculum. I’ve seen and heard of many instances of skill-and-strategy instruction being