One of the most widely-cited articles in the past 40 years on the impact of first-language vocabulary instruction on English reading comprehension is Stahl and Fairbanks (1986). Stahl and Fairbanks’ meta-analysis found that vocabulary instruction had a significant and large effect on “custom” or researcher-created texts (i.e. passages that contained the words students were taught) (Cohen’s d = .98). More important, though, was their finding that vocabulary instruction also had a significant effect on standardized reading tests (d = .30).
This latter result suggests that teaching vocabulary to students can improve reading comprehension generally, even when students are not tested on passages written to contain the words taught. This is the Holy Grail of vocabulary instruction – the “transfer” of the effects of instruction to more general measures of academic achievement.
But did Stahl and Fairbanks get it right?
Elleman and her colleagues (Elleman, Lindo, Morphy, and Compton, 2009) returned to the studies Stahl and Faribanks analyzed and found several problems with the older analysis. Most serious of these was that Stahl and Fairbanks had used effect sizes in their calculations from experiments without controls groups, then incorrectly combined them with effect sizes from studies with control groups. Stahl and Fairbanks also failed to weight effect sizes for the number of subjects in the studies (i.e. all effect sizes were treated as equal, even though some studies had only a small number of students).
Elleman et al. corrected for those errors, added both published and unpublished studies since 1986 up through the time of their review, and focused only on studies that dealt with school-age children with measures of comprehension that were at least one paragraph long. Their results were quite different from Stahl and Fairbanks’ original analysis.
Instead of an effect size of .98 for researcher-created comprehension tests that Stahl and Fairbanks found, Ellman et al. found the effect size was half of that (d = .50), although still statistically significant. For standardized reading comprehension measures, however, there was no significant impact on average for vocabulary teaching, with a very small effect size (d = .10). In fact, the researchers found only two studies that reported an increase in standardized comprehension scores after vocabulary instruction. In short, they found little evidence of “transfer.”
Even the positive finding on the effect of vocabulary instruction on researcher-created tests needs some qualification. First, effect sizes measure the differences between an experimental group and a control group, so what both groups do is important. Education research often uses a “no treatment” control group, meaning that the kids do pretty much what they “typically” do in class rather than getting the treatment.
So in the case of vocabulary instruction, one group of kids is taught the words and the other isn’t. It shouldn’t be too much of a surprise that the group taught the words does better on reading passages that contain the words they were just taught than the group that did not get taught those words. This is what is usually meant when it is said the vocabulary instruction is “effective.” It’s better than no teaching at all (and even then, only moderately better).
Second, many of the studies included in Elleman et al. gave children reading tests relatively soon after they were taught the words. We know, however, that many of the words “learned” in vocabulary instruction by children are forgotten within a few months or even weeks. Thus while “teaching to the text” may help in today’s reading passage, it may be of little use next month.
Imagine a certain hint you gave a child in math that helped him do well on one particular word problem, but only when he worked on the problem soon after being given the hint. If he comes across that same problem, say, three months from now, it will be of no help at all, since he’ll likely have forgotten it (and of course would only be of help in the first place if it were exactly that same word problem).
I should point out that there is also “decay” in word knowledge when we acquire words incidentally through reading, but this loss is much less than when words are taught explicitly (see McQuillan, 2016, for a review of the evidence from second language studies).
Third, Elleman et al. found that the more time spent on instruction, the lower the effect size or impact of vocabulary instruction on comprehension tests. Unfortunately, the researchers did not otherwise look at how efficient vocabulary instruction was in the studies.
A few additional notes and comments:
1. Elleman and colleagues also found that studies that included more “stringent control groups (i.e. control groups provided with the target words or some type of instruction vs. no instruction)” had smaller effects (p. 26). Rarely, however, do vocabulary studies compare instruction against simply spending the time reading.
2. Elleman et al. calculated that only 55% of the variance in comprehension scores could be accounted for by improvements in vocabulary knowledge in the studies they analyzed. This is lower than other estimates of the impact of vocabulary on comprehension, but a reminder that reading comprehension is more than just knowing the words in the text. Any analysis that assumes that increasing vocabulary scores will automatically lead to an increase in reading comprehension scores is flawed – and that includes a large number of studies currently being done on “academic language.”
3. Advocates of vocabulary instruction, perhaps aware of the weakness of the evidence, have tended to respond that, in effect, it doesn’t really matter that vocabulary instruction has little effect on standardized reading measures. Instead, we are told that we must simply find more “sensitive” instruments for measuring the effects of vocabulary instruction on comprehension of the texts that are used in class (e.g. National Reading Panel, 2001, Chapter 4, pp. 26-27). No word from the NRP on how we can arrange to have teachers follow students around and give them just-in-time vocabulary instruction for every passage they are about to read (though it appears that IBM may soon have the answer).
Even if you are sympathetic to this more “ecologically valid” approach to assessment, I would point out that it is an, um, ‘interesting” position for direct instruction advocates to take. Many of the same researchers have previously insisted that the impact of other reading methods (such as phonics versus whole language in early reading) must only be measured on “scientific” standardized tests and other quantifiable measures.
References
Stahl, S. A., & Fairbanks, M. M. (1986). The effects of vocabulary instruction: A model-based meta-analysis. Review of educational research, 56(1), 72-110. doi 10.3102/00346543056001072
Follow Us!