Back in the 1980s, I spent my first year out of college working as a clerk in small clothing company (I majored in ancient history). There was a sign in the break room meant to provide a little humor to the employees’ day: “The Beatings Will Continue Until Morale Improves” (allegedly from Captain Bligh of Mutiny on the Bounty). This study is a bit like that.
Milliner (2017) studied a group of intermediate English as a Foreign Language (EFL) students (N=19) at a Japanese university who were required to read at least 250,000 words from graded readers over the course of two college semesters. Students who read the minimum number of words received 10% credit on their final grades.
The books were provided via an online service; students could choose from more than 500 graded readers at various levels. Students read the books on their smartphones – a common practice in Japan, according to the researcher. Nearly all of the students (17 of the 19) met that 250,000-word target, reading an average of 263,767 words. All reading was done out of class.
The online software tracked the number of hours, pages, and books the students read. It appears students were also required (?) to take post-reading comprehension quizzes (Table 1, p. 52), although the quizzes don’t appear to have counted directly toward their final grade. The researcher administered a version of the Test of English for International Communication (TOEIC) exam before and after the treatment (December to December) to measure the students’ progress in English.
Students did in fact make significant gains on the TOEIC during their year of study – 38 points on the overall TOEIC test, and 29 points on the TOEIC reading section. Students read an average of 41 hours, 25 minutes over the two semesters, which works out to be a gain of 0.9 points per hour of reading on the overall TOEIC score.
The individual score gains, however, were not significantly correlated with the number of words read for either the total TOIEC score or the TOEIC reading section score (r = .07 and -.18, respectively).
Milliner’s results run counter to several studies that have found that the quantity of reading is positively correlated to reading gains on the TOEIC (Mason and Krashen, 2017) and TOEFL tests (Contantino et al., 1997; Gradman & Hanania, 1991). But there are some likely reasons why the results of this study differed from those of previous ones.
First, there was also no control group used in the study, nor were other sources of English input students may have received during the year-long study controlled for, so we can’t be sure if any of the gains were attributable to the extensive reading treatment, a point Milliner makes (p. 56).
Second, the student’s TOEIC score counted for 20% of the final grade, meaning students had a strong incentive to do other activities to improve their test scores, activities that may have proved more important than the impact of reading.
Third, and most importantly, none of the reading was “free reading” – it was all assigned, although students could choose their own texts. While assigned reading certainly can lead to gains in reading comprehension, it does not appear that very many of Milliner’s students got “lost in a book” or saw it as anything other than another box to tick. Nearly all of the subjects read close to the same amount – just over the minimum 250,000 words required for full credit (the standard deviation was 34,904 (Table 2, p. 54)).
The fact that students did the minimum also means that their text selections may have reflected the path of least resistance, encouraging them to choose relatively easy books. Milliner himself noted that electronic tracking was done in part to prevent any false reporting of the number of words read – not a sign that there was a high level of student “buy-in”: “The researcher was able to monitor post-reading quiz results and reading times to minimize the chances of student chicanery or cheating” (p. 55).
Though it may appear that the students did a lot of reading in a year’s time, in fact the total time spent reading was on average just over an hour a week during the school year, or a little more than 10 minutes a day. Compare this to Mason and Krashen’s subjects, who read on average more than three and a half hours a week in English (Table 3, p. 473), all without any credit or compulsion.
We don’t know for sure, but it is plausible that Milliner’s subjects did little more than “go through the motions” of extensive reading – flipping pages of an easy book, and reading enough to pass the test and get their grade. The problem with forced pleasure reading is that you may end up with neither pleasure nor reading.
Note: For a wonderful vignette on the dangers of “assigned pleasure reading,” check out this 2009 blog post on a popular (but scientifically unsupported) program used in K-12 schools called “Accelerated Reader.” The key here is not that the student cheated, but that author of the post (and all the other middle school students) looked for ways to game the system so as to read a little as possible. Are busy 20-year-old college students so different?
Follow Us!