P. Mitchell, N. Kemp, & P. Bryant. (2011). Variations among adults in their use of morphemic spelling rules and word-specific knowledge when spelling. Reading Research Quarterly, 46(2), 119-133.

There’s no better way to start a discussion of almost any topic related to reading and writing than a quote from Frank Smith.  Here’s one from Writing and the Writer (1994) on spelling:

It is hardly less complicated to remember that a particular animal is called a “horse” then to remember that the spelling of “horse” is h-o-r-s-e. And that is the basic way in which we remember the spelling of words that we can spell. At the risk of sounding banal, I have to point out that the way we demonstrate our knowledge of the spelling of a particular word is by saying what that spelling is. We do not pause to think about how the word sounds (unless we do not know how to spell it), nor do we hesitate over the possible application of particular spelling “rules” (unless again we do not know how to spell the word). But if we know how to spell word (or think we know), out the spelling comes. How do you spell “horse”? H-o-r-s-e. How do you spell “cart”? C-a-r-t. And that is the way it is, for all of the thousands of words that we can spell (or think we can spell). If we can spell the word, is because we have remembered the spelling. (p. 149)

And now, our study:

Background:

The researchers were attempting to determine whether their college-age subjects knew and could apply two “simple” spelling rules in English when presented with words that they had never seen before, and therefore could only spell by following a spelling rule.  To do this, they followed a very common procedure in psychological experiments of using pseudowords - made-up, “nonsense” words like kinkle and gries that look like they could be real words in English but aren’t.

The first rule involves the spelling of the /z/ sound at the end of a word.  Plurals in English are formed by adding an s, which can produce an /s/ sound as in cats or /z/ sound as in dogs (with exceptions such as dishes).  Third-person singular verbs can also produce either an /s/ sound as in walks or a /z/ sound as in runs.

But some words have a /z/ sound at the end that are spelled with se, ze, or zz, as in “please,” “freeze,” and “buzz.” These are neither plural nouns nor third-person singular verbs. Thus, the “morphological spelling rule” is that when the word is a plural noun or third-person singular verb and ends in a /z/ sound, spell it with an s; when it isn’t a plural or third-person singular, use the se, ze, or zz spelling.

The second rule tested involved the /ks/ sound.  In third-person singular verbs, it is spelled with a ks or kes as in socks, picks, and bakes. Otherwise, it is spelled with an x or xe, as in box, axe, and fix.

If you have learned (consciously) or acquired (unconsciously) these rules, you could then spell new words with the /z/ or /ks/ sound by determining whether or not they were “inflected” (in this case, plural nouns or third-person singular verbs). If you didn’t know the rule, you would have to guess and essentially perform no better than chance on a test of the rules.  The researchers attempted to see if college students would differ from their non-college-enrolled age mates in their knowledge of these “simple” rules.

Participants:

Two groups participated in the study: recent high-school graduates (mean age: 19.9) about to enter basic training in the British military (N = 205), which we’ll refer to as “nonstudents,” and a group of university students (mean age: 24.8) (N = 72), hereafter referred to as “students.”

Measures:

Both groups were administered the same measures of spelling competence.  These included:

1. A group-administered dictation test of 40 real English words from a standardized test of literacy achievement (the WRAT). Words were dictated to the subjects in increasing order of difficulty and students had to write down their answers.

2. A real word Spelling Choice test, where students were presented with 14 sentences, each with two possible spellings of the target word. These (real) English words all contained the /ks/ sound.

3.  A set of pseudoword spelling choice tests with invented or pseudowords placed in sentences that would indicate either an inflected ending (plural or third-person singular, with the –s or -ks/kes spelling) or uninflected ended (with the -se/ze/z or –x/xe spelling).  For example, subjects saw sentences such as:

 Would you like a spees/speeze?

Applying the morphological spelling rule, the correct answer to our sample sentence is “speeze,” since it is neither a plural noun nor a third-person singular verb.  If the sentence had been:

I’ll have three spees/speeze to go, please.

then the correct spelling would be “spees,” as a plural noun.

Results:

Both the nonstudent and student groups took the same tests.  Table 1 summarizes some of the results.

Table 1: Results on Real-Word and Pseudoword Spelling Tests

Measure	Non-students	Students
WRAT (Maximum = 40)	24.31 (6.37)	30.92 (3.54)
Real-Word Spelling Choice on /ks/ words (Maximum = 14)	13.95 (0.26)	14.00 (0.00)
Pseudoword Spelling Tests (Percentage scoring better than chance on 3 of 4 measures)	7.4%	82%

Standard deviations in parentheses

Both nonstudents and students scored well on the real word spelling measures.  Both groups did either perfectly or nearly so on the Spelling Choice test with the /ks/ words. Students outscored nonstudents on the WRAT test (effect size: d = 1.28). However, given that the WRAT test begins with “easy” real words to spell and moves on to more progressively difficult ones, the average nonstudent score seems to indicate that they are reasonably good spellers for common words, getting only around 20% fewer correct answers than the (somewhat older) university-educated students.

The big difference between the groups was their performance on the pseudoword or nonsense word tests, where only 7.4% of the nonstudents scored above chance on three of the four measures, compared to 82% of the college students.

ANCOVA analysis was performed, with performance on the WRAT test used as a covariate.  The effect size for the covariate was significant for both the nonstudents (partial eta squared = .19) and students (.01).  In other words, good real word spellers are good pseudoword spellers.

The researchers conclude that their results show that “many adults use word-specific, rather than morphological, spelling knowledge” (p. 130). The speculate on the reasons for the difference in performance between students and nonstudents:

 We collected our data in the years 2004-2006, and in the preceding two decades, when our participants were schoolchildren, teachers in the United Kingdom provided very little instruction about the link between morphology and spelling (Nunes & Bryant, 2006). Therefore, most people’s knowledge of these rules was self-taught and probably depended a great deal on their childhood interest in and experiences with reading and writing, which may be closely related to their later decision to go to university. (p. 130)

Mitchell et al. believe, then, that the neither group (students and nonstudents) were taught these rules in school. The students “taught” themselves these rules by means of their more extensive reading and writing growing up, and were thus able to perform better on the nonsense word tests than the nonstudents, who apparently did not teach themselves the rules.

Comments:

1. The failure to learn the morphological spelling rules in question did not seem to do the nonstudents any harm in spelling real words. Only a handful of them got even one wrong on the real word spelling test of the /ks/ rule. The nonstudents also did respectably well on the general spelling test, as noted above.  Their only real downfall came in the pseudoword spelling tests.  Is this important?

Hypothetically, the nonstudents may someday hear a word ending in a /z/ sound they have never seen in print.  And, hypothetically, they then may be forced to spell the word on a form or in an email, before having a chance to read it a sufficient number of times to acquire it by the “word-specific” route. Hypothetically, then, they would certainly be at a disadvantage. But this strikes me as a rather infrequent situation for a literate adult.

As the researchers themselves point out, studies show that most adults are very good spellers.  They are able to correctly spell almost all of the words they select to use in written communication. There is no crisis in adult spelling, and never has been.

2. The authors state the neither group is likely to have been taught the rules in school. The clear implication, especially for the nonstudents, is if they had been taught the rules, they would have done better on pseudoword tests. The evidence on the effectiveness of spelling instruction, however, casts considerable doubt on this assumption (Krashen, 1989; 2004). Spelling instruction is often no more effective than no instruction, and the effects tend to fade over time (Hammill, Larsen, & McNutt, 1977).

There is no good reason to believe that, had U.K. teachers spent their time teaching morphological spelling rules, the nonstudent group would have done any better. The fact that the good spellers were also the good nonsense word spellers suggests that both competencies are acquired from a single source, and that source is almost certainly reading. The burden of proof here is on the researchers to show that teaching schoolchildren morphological spelling rules would make a significant difference in the spelling competence of adults. (And remember that these were considered “simple” rules by the researchers, ones which I had to read several times to make sure my explanation of them for this post was correct!)

3. The subjects who successfully spelled the nonsense words clearly had to resort to a rule, whether consciously learned (unlikely) or unconsciously acquired through reading.  But what about the students’ real word spelling? The fact that the rules were known and applied on the pseudoword test does not mean that they were used in the case of real words, where “word-specific” acquisition could have been the source of spelling competence for the students just as much as it was for the nonstudents.

It may be that all spelling related to /z/- and /ks/-sound words in the study – and perhaps most all of our spelling competence, period – comes from “word-specific” knowledge obtained via reading, and not from the application of morphological spelling rules.  When we do acquire metalinguistic rules on top of that word-specific knowledge, such rules may only be invoked in those cases where word-specific knowledge fails us, which, for most literate adults, would happen mostly on tests with pseudowords.

To put it another way, the rules that you need to spell nonsense words are primarily useful for spelling nonsense words.

Works Cited

Hammill, D., S. Larsen, & G. McNutt. (1977). The effect of spelling instruction: A preliminary study. Elementary School Journal, 78, 67-72.

Krashen, S. (1989). We acquire vocabulary and spelling by reading: Additional evidence for the Input Hypothesis. Modern Language Journal, 73(4), 440-464.

Krashen, S. (2004). The Power of Reading: Insights from the Research. 2nd edition. Portsmouth, NH:Heinemann.

Smith, F. (1994). Writing and the Writer. 2nd edition. Lawrence Erlbaum Associates Hillsdale New Jersey

Hill, M., & Laufer, B. (2003). Type of task, time-on-task and electronic dictionaries in incidental vocabulary acquisition. International Review of Applied Linguistics, 41(2), 87–106.

I recently came across a reference to a claim related to vocabulary acquisition that struck me as rather usual: Hill & Laufer (2003) stated that a second language reader would need to read 8,000,000 words in order to acquire a mere 2,000 words.  Schmitt (2008) repeats this claim to bolster his argument that developing a vocabulary sufficient to read native-level texts requires a strong dose of direct instruction.  It is worth quoting at length from the original article:

Studies on vocabulary acquisition from reading (without any enhancement tasks) show that pick up rates of unfamiliar words range from 1–5 words in a text of over 1,000 words (Zahar et al. 2001; Luppescu and Day 1993; Hulstijn 1992; Knight 1994; Paribakht and Wesche 1993). Similar gains occur during reading books. In Horst et al.’s (1998) experiment, an average of five words were gained from the reading of a simplified version of The Mayor of Casterbridge, a text of 21,000 words. Lahav (1996) conducted a study with students who read four simplified readers, each one of about 20,000 words, and found an average learning rate of 3–4 words per book. At this rate of growth, a second language learner would have to read in excess of eight million words of texts, or about 420 novels to increase their vocabulary by 2,000 words. This would appear to be a daunting and time consuming means of vocabulary development. It is therefore reasonable that L2 learners acquire their vocabulary not only from input, be it reading or listening, but also through word-focused activities. (p. 88, emphasis added)

How did Hill and Laufer arrive at this conclusion, and is their estimate correct?

The authors give the example of Horst et al.’s (1998) study of students reading a simplified reader version of The Mayor of Casterbridge (the Lahav study mentioned is unpublished). In Horst et al., the researchers tested students on a set of 45 words. The 45 words included eight that occurred seven or more times in the text but, because they were not part of a list of high-frequency words students studied in another aspect of their language course, were likely to be unknown to the subjects.  The rest of the words were randomly selected from among low- and medium-frequency words, occurring in the text six or fewer times.

A pretest determined that the students already knew about half of the 45 target words, so the average number of new words students could have acquired on the test that was administered to them was about 23.  After reading the novel, students took the post-test, which showed an average gain of around five words out of the 23 or so words that were new to the students.  The simplified version novel they read contained a little more than 20,000.  Therefore, Hill and Laufer concluded, you can only expect to pick up around five words for every 20,000 words you read. Based on that data, they arrived at the figure 8,000,000 words that would need to be read to acquire 2,000 words: If you acquire 5 words for every 20,000 words you read, then you would need to read 400 of these 20,000 word texts (2,000/5 = 400), or 8,000,000 words.

The error here should be clear: Horst et al. did not find that only five words were acquired by students after reading a 20,000 page book, but rather that subjects got five words correct on the test administered to the them. Hill and Laufer appear to have confused the population of all the unknown words in the text with the sample of words that were included in the test. (Krashen (2004) makes a similar point about this study.) In the Horst et al., the 45 words on the test were a sample of all of the potentially new words in the text. The researchers estimated that there were about 222 words (technically, word families) that their EFL subjects might not know. They then eliminated any word that appeared only once, which left them with 75 words, and then sampled 45 words from that list to create their test. Overall, their subjects acquired a respectable 22% of the new words they encountered, as measured on an immediate post-test (administered right after the actual reading).

Although the researchers felt that words that occur only once in a text were not good candidates for being acquired, other, later research has found that while the pick up rate for such words is low, it isn’t trivial, either.  Pellicer-Sanchez and Schmitt (2010) found that the meaning of 29% of the words that appeared only once in the text they used to measure incidental acquisition were recognized by their subjects on a post-test. Waring and Takaki’s (2003) subjects recognized the meaning of 16% of the words occurring once on an immediate post-test similar to Horst et al.’s.

Even if we assume that Horst et al.’s subjects already knew half of the untested words (as they knew half of the 45 tested words), and that the pick up rate were only 10%, that would leave around nine additional words acquired, effectively tripling the total number of words acquired (222 total words – 45 test words = 177 untested words, divided by 2 = 88.5 , multiplied by .10 = 8.5 words).

There are other potential problems with the Horst et al. study (again, see Krashen (2004)), and indeed with many attempts at estimating the number of words that can be acquired incidentally through reading. In any case, the study does not provide evidence for Hill and Laufer’s claim that we need to read 8,000,000 words in order to acquire 2,000 words.

An unrelated note: Hill and Laufer has been cited recently as a study comparing explicit vocabulary instruction with incidental acquisition (File & Adams, 2010). It is not. It is instead a study of different methods of dictionary use and post-reading questions; there is no “reading only” comparison included.

Works Cited

File, K.A., & R. Adams. (2010). Should vocabulary instruction be integrated or isolated? TESOL Quarterly, 44(2), 222-249.

Horst, M., T. Cobb, & P. Meara. (1998). Beyond A Clockwork Orange: Acquiring second language vocabulary through reading. Reading in a Foreign Language, 11(2), 207-223.

Krashen, S. (2004). The Power of Reading: Insights from the Research. 2nd edition. Portsmouth, NH:Heinemann.

Pellicer-Sanchez, A., & N. Schmitt  (2010). Incidental vocabulary acquisition from an authentic novel: Do Things Fall Apart? Reading in a Foreign Language, 22(1), 31-55.

Schmitt, N. (2008). Instructed second language vocabulary learning. Language Teaching Research 12(3), 329–363.

Waring, R., &  M. Takaki. (2003).  At what rate do learners learn and retain new vocabulary from reading a graded reader? Reading in a Foreign Language, 15(2), 130-163.

There are at least three ways you could define popular when it comes to language study:

• Number of students enrolled in classes
• Rate of growth of enrolled students from one year to the next
• Ratio of native speakers to students studying the language

The first two are fairly straightforward.  The third one, what we might call a “best represented” index, consists of the ratio of native speakers of a language to the number of students studying it.  This gives us a rough indicator of how popular a language is in school versus in the “real world.”

The best data I could find on U.S. college foreign or modern language enrollements is Furman, Goldberg, and Lusin (2010), done on behalf of the Modern Language Association.  Table 1 shows winners of the first two measures of popularity, number of students studying the language and the rate of growth (here using the past 3 years, since the last MLA survey).

Table 1: Foreign Language Enrollments and Growth Rates at U.S. Colleges

Language	Students	Growth 2006-2009
Spanish	864986	5.10%
French	216419	4.80%
German	96349	2.20%
ASL*	91763	16.40%
Italian	80752	3.00%
Japanese	73434	10.30%
Chinese	60976	18.20%
Arabic	35083	46.30%
Latin	32606	1.30%
Russian	26883	8.20%
Ancient Greek	20695	-9.40%
Biblical Hebrew	13807	-2.40%
Portuguese	11371	10.80%
Korean	8511	19.10%
Modern Hebrew	8245	-14.20%
Other Languages	40747	20.80%

*ASL = American Sign Language

As you can see, Spanish has the most students, to no one’s surprise. Chinese is less popular than six other languages.  There are more than 13 times as many students studying Spanish, three and half times as many studying French, and 1.5 times as many who study German compared to Chinese.  In fact, there are more students of three dead languages (Latin, Ancient Greek, and Biblical Hebrew) than there are Chinese students (67,108 versus 60,976).

The MLA’s study also includes the percentage that each language has grown in popularity in the past three years.  Even here, Chinese is only third on the list.  Leading the pack is Arabic at 46%, followed by Korean at 19%, Chinese at 18%, and ASL at 16%.

In Table 2, using the same data from Table 1 (but excluding classical languages), I’ve added the best estimates (according to Wikipedia) of all the modern languages on the list, then provided the number of native speakers per U.S. college student of that language. This gives us the “best-represented” languages.

Table 2: Native Speakers to Student Ratio for Modern Languages in U.S. Colleges

Language	Students	Speakers	Speakers to Student Ratio
ASL	91763	517000	5.63
Spanish	864986	500000000	578.04
French	216419	128000000	591.45
Modern Hebrew	8245	5300000	642.81
Italian	80752	62000000	767.78
German	96349	101000000	1048.27
Japanese	73434	123000000	1674.97
Korean	8511	72000000	8459.64
Russian	26883	250000000	9299.56
Arabic	35083	452000000	12883.73
Portuguese	11371	220000000	19347.46
Chinese	60976	1200000000	19679.87

The best-represented language is, by far, American Sign Language.  The estimate of ASL users is, however, very approximate, since no good recent estimate seems to be available.  Even if the ASL estimate were off by 100%, however, it is clear that there are far and away more students studying ASL relative to speakers than any other language.

Granted, we are using the native speaker count from the world population for the other languages, not just the United States, so the comparison is biased in that way.  We should probably calculate the ratio by including all sign language users, not just ASL, but these figures are even harder to estimate. We can go in the other direction and use just native speaker figures for the United States to make the comparison. For example, there are around 37 million native Spanish speakers in the U.S., which would mean the Spanish ratio would drop all the way down to 43 speakers per student – not quite as low as ASL, but closer.

After ASL, Spanish is the next best represented language relative to native speakers at the international level, followed by French, Modern Hebrew, and Italian.  Chinese is dead last, though to be fair, to be at the same ratio as Spanish (578:1), there would need to be 2,076,000 Chinese students, roughly 27% greater than the entire population of college students studying languages in the U.S.

There are three languages with more than 100 million native speakers that don’t even appear on our list: Hindi-Urdu (480 million speakers), Bengali (250 million), and Punjabi (109 million).

Works Cited

Furman, N., D. Goldberg, & N. Lusin (2010). Enrollments in Languages Other Than English in United States Institutions of Higher Education, Fall 2009. New York: Modern Language Association.

Frishkoff, G., K Collins-Thompson, C. Perfetti, and J. Callan. (2008).  Measuring incremental changes in word knowledge: Experimental validation and implications for learning and assessment.  Behavior Research Methods, 40(4), 907-925.

We have long known that we acquire most of our vocabulary, in both a first and second language, through reading (Krashen, 1989).  Our knowledge of new words comes both incrementally (little by little) and incidentally (as a by-product of our main activity, comprehension).  Various methods have been developed to measure that incremental growth, especially the partial knowledge that we may gain about a word, but that falls short of a full definition of the word or even an acceptable synonym.  Below the threshold of being able to translate (in the case of second languages) or give a dictionary definition of a word, it appears that there exists a wealth of other, previously undetected word knowledge that readers are acquiring.  The study I review here introduces a new method for capturing that vocabulary growth.

Subjects: U.S. university students (N = 21), all native speakers of English.

Treatment: Subjects were given 60 unknown words in English to read in sentences.  The words were very rare and unlikely to be known by the subjects. Each word was seen six times in different sentences, in either useful or “good” contexts or misleading or “bad” contexts.  A good context would help the reader determine the meaning of the word, such the word “abrogate” in this example:

This system has been weakened since 1983, and the current Liberal party government seeks to further weaken or abrogate it.

“Bad” context (so named by the researchers) were sentences in which the target word was used in a context that was appropriate for another, similar sounding and similarly spelled distractor word.  For example, the target word “abrogate” would be used in place of a distractor, “arrogate,” as a malapropism:

Traditional distributors…abrogate to themselves the role of determining what’s proper for their customers to read.

In a bad context, the target word is misused as a replacement for the distractor word, thus potentially misleading the reader into inferring an incorrect meaning.

Some of the target or tested words were presented in all “good” contexts, some with three good and three bad contexts, and some with one bad and five good contexts.  Accuracy scores were predicted to vary according to the “goodness” of the six contexts.

Measures:  Subjects were given a “synonym judgment test” before and after the experiment, in which they were provided the 60 target words and asked to select the best synonym for each.  Among the possible answers was a synonym for the distractor word, so the researchers could test just how far the bad contexts would lead readers astray. The synonym judgment test provided a pre/post measure of word knowledge gain due to the experimental phase of the study, in which, as noted, subjects actually read sentences containing the target words in good and/or bad contexts.

In addition, after each exposure to the target word in context, two additional measures of words knowledge were taken.  First, subjects also had to judge the semantic appropriateness of sample sentences that contained the target words. While not asked to give a definition, subjects had to understand enough of the word to determine if the test sentence using the target word made sense or not.

Second, subjects wrote down the meaning of the word (their best guess) after reading each sentence.  These responses were then analyzed using a recently developed method of statistical modeling called the “Markov estimation of semantic association” (MESA) to measure small, incremental growth in word knowledge after each exposure. Each guess the subjects gave as to the meaning of the target word was given a score indicating its “distance” from the correct definition. The MESA model used a variety of factors to determine distance, such stemming (words based on a common morphology), synonymy (words of similar meaning), co-occurrence (words that tend to appear together in the same context, such as “election” and “politics”), and associative strength (words one might give if doing a “free association” with the term), among others.

If the subject’s response had no possible links to a correct answer along any of the dimensions that were included in the model, knowledge was scored -1; a correct definition or synonym was scored 0.  This allowed the generated answers to be placed on a continuous scale marking their approaching accuracy to the word’s correct meaning, and thus revealing incremental growth that fell short of full knowledge of the word.

Here’s an example of how responses might be scored for the word abditive (meaning “hidden”) presented in all “good” contexts.  Remember that responses are more accurate as they approach zero on the MESA scale (taken from Example 1, p. 917):

Trial	Response	MESA Score
1	Avoidant	-1.00
2	Attitude	-.83
3	Sneaky	-.50
4	Secretive	-.35
5	Secretive	-.35
6	Hidden	-.35

Results:  Words found in more useful contexts were, not surprisingly, acquired more quickly than those placed in “bad” or misleading contexts.  The average gain from pretest to post-test on the synonym test was about 12%, which means on average subjects picked up an accurate meaning of around seven of the 60 target words after six exposures.  This 12% figure appears to be for all context conditions combined, however, including the misleading ones.  The article’s Figure 2 (a) (p. 915) shows that when words are presented in all good contexts, the “pickup” rate was closer to 17%.  Subjects also were less likely to choose the distractor’s synonym when the word was presented in good contexts, showing another aspect of word knowledge growth.  Notice in the example above how each guess captures something of the meaning of the word, some aspect that shows the reader closing in on the real definition.

The MESA measure confirmed that words presented in good contexts were more easily acquired than those in bad contexts. More importantly, the MESA measure showed an incremental, linear pattern of more accurate word knowledge after each “good” context exposure, indicating that each guess was getting nearer and nearer to the correct definition.

Comments:

1. The important thing about this study is the methodology for detecting small changes in word knowledge after each exposure.  Frishkoff et al. have shown that even when subjects are asked to produce a meaning of target words encountered incidentally (versus merely to recognize the correct meaning in a multiple choice format), there is evidence of considerable cumulative growth in knowledge, knowledge that often falls short of the “correct” answer.

The measurement of the acquisition of partial word knowledge is by no means a new thing. The MESA methodology is merely a more sophisticated approach to the problem, and a reminder that many current measures will likely miss increases in vocabulary growth that “fly under the radar” of a translation or even a receptive measure such as a multiple-choice test, and thereby underestimate the incidental growth in word knowledge through reading.

2. The study’s finding that subjects picked up 12-17% of words presented in good or mostly good contexts is in line with the rates of “pickup” found in studies of incidental acquisition in an L2 setting. But this figure is, as already indicated, only part of the story.  The true amount of new knowledge gained incidentally of unknown words is much greater, as the MESA scores demonstrate. In the case of the Frishkoff et al. subjects, something was probably being picked about several of the words that appeared in good or mostly good contexts, making the real gain from reading much more than the 12-17% that passed the threshold of being “known” on the post-test.

The MESA measure is an interesting advance in assessing incidental acquisition, helping us detect the previously unseen incremental growth that takes place even after a single pass or two at a word. This also means that current approaches used in vocabulary acquisition research likely underestimate the effect exposure has on word acquisition.

The case for incidental acquisition through reading keeps getting stronger and stronger.

Works Cited

Krashen, S. (1989). We acquire vocabulary and spelling by reading: Additional evidence for the Input Hypothesis. Modern Language Journal, 73(4), 440-464.

by Stephen Krashen 

I comment on two statements in K. Nielson’s paper, “Self-study with language learning software in the workplace: What happens?” published in Language Learning and Technology, 2011, 15 (3): 110-129.

The first is the claim that Rosetta Stone is based on my work. Nielson cites Saury (1998), who cites promotional literature from Rosetta Stone saying that Rosetta Stone is based on the comprehension approach. I was not aware of this until Nielson’s paper brought it to my attention. I must point out that I have had no connection of any kind with Rosetta Stone. I played no role in developing its approach, nor have I analyzed it. I do not know if it is in reality consistent with my work. I am not responsible for Rosetta Stone’s failures or successes.

The second concerns Nielson’s statement that my work has “fallen out of favor in more recent SLA research” but provides no clear details or any citations. This statement violates a core academic principle of providing empirical support for claims.

I have attempted to respond to every empirical criticism of my positions since the 1970‘s and continue to do so. I have published far too many responses to list here, but the following are some of my responses published in the last decade (see also papers available at www.sdkrashen.com):

Krashen, S. 2003. Explorations in Language Acquisition and Use: The Taipei Lectures. Portsmouth, NH: Heinemann.

Krashen, S. 2004. False claims about literacy development. Educational Leadership 61(6): 18-21.

Krashen, S. 2004. Skyrocketing scores: An urban legend. Educational Leadership 62(4): 37-39.

Krashen, S. 2005 Is In-School Free Reading Good for Children? Why the National Reading Panel Report is (Still) Wrong Phi Delta Kappan 86(6): 444-447.

Krashen, S. 2009. The Comprehension Hypothesis extended. In T. Piske and M. Young-Scholten (Eds.) Input Matters in SLA. Bristol: Multilingual Matters. pp. 81-94.

Krashen, S. 2009. Does intensive reading instruction contribute to reading comprehension? Knowledge Quest 37 (4): 72-74.

Krashen, S. 2010. The Goodman-Smith hypothesis, the input hypothesis, the comprehension hypothesis and the (even stronger) case for free voluntary reading. In P. Anders (Ed.), Defying Convention, Inventing the Future in Literacy Research and Practice: Essays in Tribute to Ken and Yetta Goodman. New York: Routledge.

Krashen, S. 2011. A note on error correction: The effect of removing one outlier in Ryoo (2007). International Journal of Foreign Language Teaching 6(1): 5-6.

Krashen, S. 2011. Incidental acquisition of spelling competence: A re-analysis of Pérez Canado (2006). International Journal of Foreign Language Teaching 6(1): 15-24.

References

Saury, R.E. (1998). Creating a psychological foundation for the evaluation of pre-packaged software in second language learning. Proceedings of ED-MEDIA/ED-TELECOM 98 World Conference on Educational Telecommunications, Freiburg, Germany.  Available through ERIC.

Evans, N., K.J. Hartshorn, R. McCollum, & M. Wolfersberger. (2010). Contextualizing corrective feedback in second language writing pedagogy. Language Teaching Research, 14(4), 445 -463.

Hartshorn, K.J., N. Evans, P. Merrill, R. Sudweeks, D Strong-Krause, & N. Anderson. (2010). Effects of dynamic corrective feedback on ESL writing accuracy. TESOL Quarterly, 44(1), 84-109.

Evans, N., K.J. Hartshorn, and D. Strong-Krause. (2010). The efficacy of dynamic written corrective feedback for university-matriculated ESL learners accuracy. System, 39, 229-239.

In the world of language teaching, some ideas die hard.  Error correction is one of those ideas that keeps rising from the dead after each seemingly fatal blow.  Krashen’s 1984 book, Writing, reviewed the evidence on error correction and found that it rarely did what it promised to do – improve students’ grammatical accuracy.  Subsequent papers by Krashen and a series of excellent critiques by John Truscott  (1996, 1999, 2004) put additional nails in the coffin. But error correction lives on, with plenty of defenders in the applied linguistics field.

I look here at three recent studies from researchers at Bingham Young University that take error correction to its logical conclusion – correcting nearly every mistake students make in every essay throughout an entire semester.  None of the studies, in my view, show that error correction was worth the massive effort that was invested in it.  I start with descriptions of the studies, then comment on them below.

Evans, Hartshorn, McCollum, & Wolfersberger (2010)

Participants: Two groups ESL students described as “advanced low” studying in an intensive English program (IEP) for a 13-week semester.  (This was apparently the pilot study for the next two studies, and contained no control group.)  The groups (N = 12 and 15, respectively) were from different academic semesters, Winter 2007 and Summer 2007, and students ranged in age from 18 to 33.

Treatment: There were six steps in the rather lengthy error correction treatment, which the authors refer to as “dynamic Written Corrective Feedback” (WCF):

1. Each group wrote 10-minute, timed essays at the beginning of most class sessions (the class met Monday through Thursday).  They were told to “follow the conventions of good paragraph writing, be as linguistically accurate as possible, and make the content substantive” (p. 455). Topics were assigned for each essay.  On average, the groups in this study wrote 31 paragraphs throughout the semester.
2. The teacher collected the essays and then corrected them, returning them the next class session to the students.  Written feedback consisted of “marking the papers for lexical and syntactic accuracy” using 20 error-correction symbols.  Citing work by Ferris, the researchers state that the teacher indicated but did not correct “errors students can treat – those that can be corrected with systematic grammar rules” with one of the 20 symbols. But the teacher also did “direct error correction,” actually correcting the errors that students were thought unable to treat, described as “those that result from aspects of the language that must be acquired over time, such as prepositions or some lexical features” (p. 455). Then, the teacher assigned a “holistic” grade, with 75% for accuracy and 25% for content.
3. The coded/corrected paper was returned to the student during the next session.  The student, in turn, had several additional tasks:
   (a) Keep a tally of errors by type of error;
   (b) Keep a list of all errors committed in context by category, which consisted of errors “typed exactly as they were originally and erroneously written” (p. 455);
   (c) Highlight or underline each error on their typed list of errors; and finally
   (d) Edit, type, and resubmit the paragraph to the teacher for a second review.
4. The teacher then marked the typed paragraphs again for accuracy, but this time indicating the place where the error occurred with a check mark, a circle, or by underlining, although the teacher could, if needed, also supply the error code again. The papers were then returned to the students.
5. Students corrected (if necessary) their second draft and returned it to the teacher.
6. The teacher corrected and indicated the errors again, if necessary, and the process repeated until the paragraph was error free.  The researchers note that most paragraph were error free within two drafts.

Hartshorn et al. (2010): 

Participants: 47  low- and mid-advanced university Intensive English Program (IEP) students (mean age: 24 years) of various language backgrounds.

Treatment:  Students in the treatment group underwent substantially the same process of total error correction as in the previously discussed study of Evans et al.: 10-minute essays in each class, errors coded by the teacher, tallied and corrected by the student, and so on until the essay was free of errors.  In addition, the teacher discussed common errors in class.

The control group participated in a “process writing” approach during the 15 week period, during which, however, errors were also corrected. They did not do short, timed compositions like the experimental group, but wrote four multi-draft term papers and received feedback on each draft.  The total amount of writing, according to the researchers, was roughly the same.

Measure: The pretest and posttest consisted of 30-minute typed essays rather than the in-class essays.  Writing accuracy was measured by the ratio of error-free T-units to total T-units. Also measured were “rhetorical competence” with a rubric adapted from the TOEFL iBT; writing fluency, defined as the number of total words; and writing complexity, defined as the mean number of words per T-unit.

Results: The treatment and control groups did not differ significantly on the rhetorical competence, fluency, or complexity measures, although the contrast group wrote slightly more than the control group (Cohen’s d is not reported; partial eta squared was .07). The treatment group outscored the contrast group on accuracy, however, by what appears to be a wide margin.  The researchers somewhat confusingly report the results as whole numbers, although they are in fact ratios of error-free T-units to total T-units. Results are summarized in Table 2, but using percent of error-free T-units for the pre- and posttest calculated from the ratios given in study’s Table 4 (p. 99).

Table 2: Percent of Error-Free T-units for Experimental and Contrast Groups in Hartshorn et al. 

	Pretest	Posttest	Difference
Experimental	14.02% (15.0)	24.16% (19.46)	10.14
Control	16.3% (10.7)	13.78% (11.81)	-2.52

Effect size: partial eta squared = .21; Cohen’s d = .64 (calculated from Table 4, p. 99, using pooled standard deviation)

The researchers conclude that WCF was effective in significantly improving writing accuracy, noting that the effect size as measured by partial eta squared was large (using Cohen’s (1988) guidelines). Cohen’s d was not calculated in the original article. It was .64, which is considered by Cohen a medium effect. In terms of the practical significance of the results,  Hartshorn et al. note that the treatment group’s writing was “just over 75% more accurate than the writing of the contrast group.”

Evans, Hartshorn, & Strong-Krause (2010):

Participants: Both the treatment group (N = 14) and the contrast group (N = 16) were university ESL students (mean age: 21).

Treatment: Both treatment and control groups received essentially the same semester-long curriculum as the treatment and control groups of Hartshnorn et al. (2010) and Evans et al. (2010).  The treatment group received extensive error correction, including having to code, tally, and log all of their errors, as well as rewrite their 10-minute in-class essays over until there were error free, a process that the researchers note “many times…required multiple drafts” (p. 234).  The 10-minute essays were written “three to four times per week” over the 13-week semester, totally about 19 pages of “polished” writing.  The contrast group received a process-writing approach, although their errors were also corrected, but not as consistently or extensively as the treatment group.

Measure: Fluency and complexity were measured in a similar manner as in Hartshorn et al. (2010) (number of total words written and mean number of words per clause).  As in Evans, Hartshorn, McCollum, and Wolfersberger, the accuracy measure was changed from the ratio of error-free clauses over total clauses, since it was thought to provide a more sensitive measure of change than T-unit analysis.

Results: Both fluency and complexity of the treatment group was slighly worse than the contrast group, although the researchers report that the effect sizes were relatively small (partial eta squared .04 and .06, respectively; Cohen’s d was not reported). On the measure of accuracy, the treatment group outscored the comparison students.  Results are reported in Table 3, from Evans et al.’s Table 3 (p. 235).

Table 3: Percent of Error-Free Clauses for Experimental and Contrast Groups in Evans et al.

	Pretest	Posttest	Difference
Experimental	47.10% (11.2)	57.80% (10.9)	10.7
Control	51.40% (12.6)	50.30% (19.4)	-1.1

Effect size: partial eta squared = .16; Cohen’s d = .48 (calculated from Table 3, p. 235)

Comments:

1. As with many studies on error correction and “focus on form,” the conditions for Monitor use appear to be met during the elicitation measures of all three studies. The Monitor (Krashen, 1982) is our ability to use conscious knowledge (learning) of the language (such as grammar rules) to make our speaking and writing more accurate than what it might otherwise be. To use the Monitor, you must (1) know the rule you need to apply in the given situation, (2) be focused on the form or accuracy of the sentence you are producing, and (3) have time to use the rule.  These conditions are difficult to meet in the real world, but, when the conditions are met, the Monitor can improve our accuracy. In these three studies, as we would expect, the students who knew the rule, were focused on form and accuracy, and had sufficient time to use their conscious knowledge (learning) were more accurate than (in the two studies with control groups) students less focussed on form and presumably less knowledgeable of the rules.

The results of these studies, then, are perfectly consistent with current second language acquisition theory (Krashen, 2003) on the use of conscious knowledge in language production.  There is little doubt that after 13-15 weeks of massive error correction and perhaps as many as 30 hours spent focussing on form, not only in class but in their homework, students already oriented to grammar instruction and error correction (university ESL/IEP students) were able to use their conscious knowledge during the assessments.  While the essays were timed in all three studies, the 10-minute essays in the first study were very short pieces of writing in which accuracy was heavily stressed (the sample student essay included in the article is only eight sentences and less than 100 words long, although no measures of fluency were reported to know how typical this sample was). For the other two studies, 30 minutes would be for most students sufficient time to focus on the grammatical correctness of their writing.

In addition to meeting the conditions of the Monitor, the treatment groups were also very practiced in timed essay writing, having done so 30 or more times during the semester.  There is no indication that the contrast groups did any timed writings, and were thus likely to be less practiced at writing under such conditions.

2. As the researchers point out, this was not a comparison of error correction and no error correction, but (in studies 2 and 3 above) extreme corrective feedback with more traditional error correction. Other studies, such as Syying Lee’s work on extensive reading and writing (Lee & Hsu, 2009) have found accuracy improves significantly simply through more reading, without the extensive (and time-consuming) WCF used here.

3. The studies lacked any delayed post-test.  This is a crucial element in any study on the effects of form-focused instruction, as previous studies which have included a delayed post-test have nearly always found a sharp decline in the gains demonstrated on the immediate post-test. The effect of explicit instruction typically fades, although Krashen (2003) notes that it can take up to several months for the declines to show up (p. 42-43). This weakness of all three studies is alone reason to question the researchers’ optimistic assessment of the benefits of corrective feedback.

The gains that were found were indeed large by Cohen’s guidelines for partial eta squared, but, for the last two studies, somewhat less so when calculated from the mean post-test scores (d = .64, a medium to large effect for Hartshorn et al., d = .48, a small to medium effect for third study).

4. Unlike the researchers, I do not find the practical effect of this huge investment of time in error correction very impressive.  In the first study, students went from having 55-58% of their clauses with errors to 45-46% with errors. In Hartshorn et al., treatment group students went from having 86% of their T-units with errors to “only” 76% error-filled T-units.  In the third study, students in the treatment group went from having 53% of their clauses with errors to 42% with errors, all this after painstaking and massive error correction efforts.

While this is an improvement, it is dubious whether teachers would think the considerable effort involved in carrying out this “all-correction, all-the-time” agenda in their own classrooms worth it for such a small result. Remember that the teachers coded nearly every error in the essays from nearly every class period, in addition to having to re-code/correct follow-up essays. Students had to re-type the errors, tally them, classify them, underline the error, type and correct the essay, then repeat the process if the essay wasn’t error free. No estimates are given of how much time the teachers and students collectively spent on correcting errors, but is appears to be substantial. These of course are, as pointed out, students coming from a population already heavily geared toward grammar study, making them ideal candidates for this sort of treatment. To leave the semester still making so many errors can hardly be claimed as a victory for error correction.

Works Cited

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed). Hillsdale, NJ: Lawrence Erlbaum.

Krashen, S.D. (1982). Principles and practice in second language acquisition. Pergamon Press.

Krashen, S.D. (1984). Writing: Research, theory, and applications. Torrance, CA: Laredo Publishing.

Krashen, S.D. (2003). Explorations in language acquisition and use: The Taipei Lectures. Portsmouth, NH: Heinemann.

Lee, S. Y., & Hsu, Y. Y. (2009). Determining the crucial characteristics of Extensive Reading programs: The Impact of Extensive Reading on EFL writing. The International Journal of Foreign Language Teaching (IJFLT) , 12-20.

Truscott, J. (1996). The case against grammar correction in L2 writing classes. Language Learning, 46, 327-369.

Truscott, J. (1999). The case for “The case against grammar correction in L2 writing classes” A response to Ferris. Journal of Second Language Writing, 8, 111-122.

Truscott, J. (2004). Evidence and conjecture on the effects of error correction: A response to Chandler. Journal of Second Language Writing, 13, 337-343 .

Wolf, F. (1986). Meta-analysis: Quantitative methods for research synthesis. Newbury Park, CA: Sage Publications.

 Nielson, K. (2011). Self-study with language learning software in the workplace: What happens? Language Learning & Technology, 15(3), 110-129.

Almost no one who studies a foreign language in the United States gets very far.  Millions of high school and college students show up to their Spanish or Chinese I classes in the hopes of learning to communicate in a foreign language, but only a relatively small percentage will ever do so. Hundreds of thousands of adults pay good money for language CDs, Dummies guides, and fancy software, but with equal rates of failure.

None of this, sadly, is news.  The language teaching profession has known about the severity of foreign language (FL) “dropout” rates for decades.  The study I review today adds to the rather depressing body count of foreign language study by examining the attrition rates for two popular self-study programs.

Subjects: Two groups of U.S. government (USG) employees working in agencies that provide self-study language training were studied.  Group I (N = 150) consisted of USG employees from a number of different agencies.  All were absolute beginners in the language they choose to study (Arabic, Chinese, or Spanish).  In Group II (N = 176), students were all employed by the U.S. Coast Guard.  Group II only studied Spanish, but unlike Group I, were at various proficiency levels.  All subjects were volunteers who sought to participate in the study, and Group II students were given time off their regular duties to do so (three hours per week).

Treatment: Group I used the popular commercial program Rosetta Stone (RS), an Internet-based software program designed for the self-study of languages. While the program is available on CD, all participants could only access an online version, per the procedure of the participating USG agencies. Group II used Aurolog’s Tell Me More (ATTM) software, also available only online.

Group I students agreed to use the RS materials online for 10 hours per week for 20 weeks, giving them time to complete the recommended 200 hours for Level I of the courseware.  Group II students agreed to use the ATTM Spanish courseware for at least five hours per week for 26 weeks.

Measures: Group I students were given proficiency interviews over the phone in which they were asked to identify and describe pictures similar to the ones that appeared in their RS course. The tests were administered after each 50 hour segment of the 200-hour long study period. They were also given an ACTFL Oral Proficiency Interview (OPI) as an exit test.

Group II students took the ATTM placement and exit test, as well as the Versant for Spanish oral proficiency assessment, which correlates highly with the OPI exam. Group II students who knew some Spanish already were given the Versant as a pre-test, and all students were to be given it as a post-test.

All students were asked to keep a “learner log” to track how much time they studied with the materials.

Results:  Nielson summaries her results this way: “The most striking finding [for both groups]…was severe attrition in participation” (p. 116).

Table 1 summarizes her data from both groups, and the steep dropout rates for both programs are evident. Nielson used different categories in reporting the data for the RS and ATTM groups (as noted in Activity column of Table 1), but the pattern is very clear.  In addition to the raw numbers, I’ve put the percentage of the “surviving” students of the total who originally enrolled in the program (Step 1).

Table 1: Attrition in Participation Using the Rosetta Stone and Tell Me More Software Programs 

Activity (Rosetta Stone / ATTM)	Rosetta Stone	% Survivors	Tell Me More	% Survivors
1. Volunteered and Signed Consent Forms	150	/	176	/
2. Obtained an RS Online Account / Took ATTM Placement Test	120	80%	103	58.50%
3. Actually Accessed RS Account / Used ATTM for 5 hours	73	49%	61	35%
4. Spent More than 10 Hours Using Course	32	21%	17	9.60%
5. Complete 1st RS Assessment (50 hours) / Used ATTM 15-25 hours	21	14%	9	5%
6. Completed 2nd RS Assessment (100 hours) / Used ATTM 25 hours or more	6	4%	7	4%
7. Completed Final Assessments and OPI / ATTM Exit Test	1	0.60%	4	2.20%

From Nielson’s Tables 1 and 2, pp. 116-117

Of the students who signed up for the RS courses, only 21% completed even 10 hours of the 200 hour course (5% of the total). Only one of the 150 volunteers made it to the end.  For the ATTM course, less than 10% made it to the 10-hour mark (13% of the way through the course), with a mere four completing the final assessment.  The attrition rate from beginning to end was 99.4% for Rosetta Stone, and 97.8% for ATTM.

The language proficiency assessments were taken by only a fraction of the participants. In general, the researcher reports that more hours spent with the course did produce better scores on the interim assessments.  The number of subjects who took the exams was for the most part too small to be of much use in evaluating the effectiveness of the programs themselves.

Comments:

1. Nielson’s results should be rather disappointing to the USG officials who paid for their employees to access these courses, not to mention the taxpayers. She does report that not all of the attrition can be blamed on the programs themselves, however. A significant percentage of the students apparently had a variety of technical problems with the software (browser plugins that would not load, system crashes, etc.).  Some of the participants reported dropping out due to being assigned overseas during the course of the study, not having enough time, or having experienced some change in their work situations.  Most, however, did not provide reasons for dropping out.  In addition to the technological problems, there were also complaints about the content of the courses themselves.

While one can blame technology for part of the attrition in Nielson’s study, no such excuse can be offered for students studying with traditional paper textbooks.  In the only other study of attrition in using self-study materials that I am aware of (McQuillan, 2008), I came to very similar conclusions as Nielson: hardly anyone finishes a self-study foreign language course.  In my study, I looked at how public library materials were used by patrons at a local library, using a “Wear-and-Tear” index to determine how far along patrons had gone in the books, indicated by worn pages, dog-eared pages, etc.  On average, those who checked out the language courses (only books were analyzed) got no further than 17% through the text before abandoning them.  Taking these results with Nielson’s data, we can conclude that attrition rates for introductory self-study materials approach 100%.

2. Nielson is rightly critical of the huge dropout rates for the software programs she studied, and notes that such self-study products are “unlikely to work by themselves,” without proper support (p. 125).  I agree, but we should also keep in mind that students often don’t fare much better in traditional classroom language classes.  Dropout rates are also quite high, and those who survive, as Dupuy and Krashen (1998) discovered, are mostly those who have studied abroad, not those who have come up through the ranks of the FL courses themselves (in Dupuy and Krashen’s sample, an astonishing 84.5% of the students in upper-division courses had studied abroad!).

Attrition in FL classes is not a new phenomenon.  Table 2 below shows data from Coleman (1930) on statewide high school foreign language enrollments by level for an unnamed northeastern U.S. state in 1925, as well as more recent data for all 50 states from the year 2000 (Draper & Hicks, 2002).  As in the case of Nielson’s data in Table 1, in addition to the raw figures, I’ve put the percentage of the Level I students who survived each passing year of study.

Table 2: Foreign Language Enrollments by Level in High School in 1925 (1 State) and 2000 (50 states)

	Total Enrollment, 1925	% of Year 1 Enrollment	Total Enrollment, 2000	% of Year 1 Enrollment
Level I	3,594	/	1,133,626	/
Level II	2,839	79%	797,800	70%
Level III	364	10%	346,200	31%
Level IV	160	4.50%	201,805	18%

For 2000 data, includes any higher levels (e.g. Spanish V or VI) plus Advanced Placement courses. Data from Coleman (1930) and Draper & Hicks (2002)

We see the similar declines from Level I (freshman) to Level IV (seniors), despite the difference of 75 years in the data.  The attrition rate for 1925 was 96.5%; for 2000, it was only marginally better at 82%.

The situation does not improve at the college level.  Furman, Goldbert, and Lusin (2007) report that of the 1,536,614 undergraduates enrolled in the top 15 foreign languages at U.S. colleges in 2006, only 17% were enrolled in upper-division courses. This attrition rate is similar to what we find at the high school level.

There may be several reasons why the dropout rates are so high in FL classes.  No doubt most students take lower-level courses to meet course requirements, and abandon them once they do so.  But my own experience as an FL teacher in high school and at the university is that many of these same students, if asked in their Level I courses, would indicate a real willingness to acquire the language they’re studying.

At least part of the blame for these high rates of attrition lies with poor teaching methodology.  Tse (2000) cites research from the 1970s showing that a significant percentage of students found their FL classes “unstimulating and uninteresting” (p. 72). Are we much better off today?

Research Cited

Coleman, A. (1930). A new approach to practice in reading a modern language. Modern Language Journal, 15(2), 101-118.

Draper, J., & J. Hicks.  (2002). Foreign language enrollments in secondary schools, fall 2000. Alexandria, VA: American Council on the Teaching of Foreign Languages.

Dupuy, B., & Krashen, S.  (1998). From lower-division to upper-division foreign language classes: Obstacles to reaching the promised land.  ITL: Review of Applied Linguistics, 119/120, 1-7.

McQuillan, J.  (2008). Does anyone finish the Berlitz tapes?: A novel method of perseverance for commercial language courses.  International Journal of Foreign Language Teaching, 4(1), 2-5.

Furman, N., D. Goldberg, & N. Lusin.  (2007).  Enrollments in languages other than English in United States institutions of higher education, Fall 2006.  New York: Modern Language Association.

Tse, L. (2000).  Student perceptions of foreign language study: A qualitative analysis for foreign language autobiographies. Modern Language Journal, 84(1), 69-84.

Abstract

Three multivariate analyses, all controlling for the effects of poverty, confirm the importance of the library. Replicating McQuillan’s analysis of 1992 NAEP scores, access to books in school and public libraries was a significant predictor of 2007 fourth grade NAEP reading scores, as well as the difference between grade 4 and grade 8 2007 NAEP reading scores, suggesting that access is important for improvement after grade 4. Access (school/classroom libraries) was a significant predictor of scores on the PIRLS test, a reading test given to fourth graders in 40 countries.

PDF version with Appendix

Article

It has been firmly established that more access to books results in more reading and more reading leads to better literacy development (Krashen, 2004).

It is thus reasonable to hypothesize that more access means better reading.  This prediction has been confirmed by a number of studies showing a positive relationship between library quality and reading achievement (McQuillan, 1998; Lance, 2004, and studies reviewed in Krashen, 2004)

In a multivariate study, McQuillan (1998) examined the relationship between access to reading material and scores on the 1992 NAEP reading test given to samples of fourth graders in 42 states in the US.  His measure of access was a combination of three measures of access to reading material at home, two of access to reading in school, and two of access to reading in the community. Table 1, a multiple regression analysis from McQuillan (1998), tells us that even after controlling for the effect of poverty, access to print was a significant and strong predictor of performance on the NAEP reading test: Those with more access did better.

The combination of poverty and print access accounted for 72%  (r2 = .72) of the variability on the NAEP, that is, if we know the level of poverty of families in a state, and how much reading material is available to children in that state, we have 72% of the information we need to predict how well fourth graders in that state scored on the NAEP.

Table 1: Predictors of NAEP reading test scores, grade 4, 1992, 42 states

Predictors	beta	t	p
Poverty	-0.45	5.07	.01
Print Access	1.12	4.3	.01

r2 = .7

The goal of this paper is to report some recent progress in this area, using multivariate analysis.

A Replication

Table 2 presents a replication of McQuillan’s findings using the 2007 fourth grade NAEP and more recent measures of poverty and access to books (a combination of books per student in school libraries and per capita total circulation in public libraries in each state).  (Means, standard deviations, and inter-correlations among the variables are presented in the Appendix, tables A1 and A2.)  This analysis controls for the presence of English learners by only including scores for fluent English proficient children.  Once again poverty is a strong predictor of scores, and once again access to books makes an independent contribution to reading achievement.

Table 2: Predictors of NAEP grade 4, 2007, 51 states

Predictors	beta	t	p
poverty	-0.72	7.42	0
access	0.53	1.62	0.055

r2 = .65;  adjusted r2 = .63 Fluent English proficient students only

The Grade 4 to 8 Difference

A separate analysis was performed to try to determine what factors are responsible for improvement after grade 4, or, more accurately in this case, the difference between grade 4 and grade 8 scores.  This multiple regression analysis is presented in table 3. This analysis indicates that, not surprisingly, that grade 4 scores are a strong predictor of grade 8 scores. It is surprising, however, that poverty is a weak predictor of the difference between grade 4 and grade 8. Recall that the impact of poverty is strong, however, on the grade 4 test.

Table 3: Predictors of NAEP grade 8, 2007, 51 states

Predictors	beta	t	p
NAEP grade 4	0.857	10.68	0
Poverty	-0.076	0.96	0.17
Access	1.26	4.59	0

 r2 = .89  Fluent English proficient only

Of interest to us is that access to books, again a combination of school library holdings and public library circulation, is a significant predictor of the difference in NAEP reading scores between grade 4 and grade 8.

The r2 of .89 means that knowing the fourth grade NAEP scores for a state, the level of poverty, school library holdings and public library circulation is 89% of the information we need to predict a state’s grade 8 NAEP reading score.

Late intervention

The effect of poverty on fourth grade reading is enormous, but access to books can contribute to fourth grade reading, regardless of poverty. The analysis also indicates that those who read better in grade four also read better in grade eight, but access to books can help here as well.  This agrees with data showing that “late intervention” in the form of recreational reading is not only possible but can be effective (Krashen and McQuillan, 2007).

To get a more precise idea of the impact of access to books, we can analyze the increase in r2 achieved by adding access to the effect of poverty.  In grade 4, after controlling for poverty, access adds .02 to the r2, increasing our ability to predict reading scores by 2%.  Access increases our ability to predict the grade 4 to 8 difference by nearly 5%. As indicated in Table 4, both public library circulation and school library holdings contributed to these increases.

Table 4: Gains in r2 

Predictors	Access	Public Library	School Library
grade 4	2%*	1.60%	1%
difference 4-8	4.8%*	2.7%*	3%*

 * = statisically significant,  p < .10

This investigation used states of the USA as units. Our second study expands the investigation of the relationship of access to reading to the international level, with countries as units.

The PIRLS Study

PIRLS (Progress in International Reading Literacy Study) administered a reading test to fourth graders in over 40 countries.  The PIRLS test attempts to measure both reading for literary experience and reading to acquire and use information (Mullis, Martin, Kennedy, and Foy, 2007).  Students took the test in the national language of their country.

PIRLS provides not only test scores, but also the results of an extensive questionnaire given to teachers and students, including attitudes, reading behavior outside of school, and classroom practices (Mullis et. al., 2007). PIRLS also supplies data on socio-economic class. The items on the questionnaire relevant to this study and SES statistics are presented in the Appendix (Table A3).

We present here two analyses of the PIRLS data, designed to further test the impact of access to books on scores on the PIRLS reading test. The first is a complex or full analysis that included as much of the information provided by PIRLS as possible, and the second is a simpler analysis, using only selected variables. We only included countries for which complete data was available for all factors (for a list of the countries included, see Appendix Table A4).

The full (complex) analysis

In order to deal with the vast amount of information supplied by the PIRLS questionnaire, the data was factor analyzed, a statistical technique that assigns predictors into groups that behave similarly, as one factor.

Factor analysis revealed four factors: SES/home (Socio-economic status and home resources, including books in the home), Literacy (free reading of fiction, sustained silent reading in school, parental reading, parental education), Libraries (school and classroom), and Instructional Factors. (Inter-correlations are in Table A5 of the Appendix and details of the factor analysis are presented in Table A6 of the Appendix.)

The Library factor was by far the strongest predictor in the multiple regression analysis. The Literacy (free reading) factor was positively related to reading scores but did not reach statistical significance. Although the SES/home factor correlated highly with reading scores (r = .64; see table A5 in the appendix), the SES/home factor was not a significant predictor of reading scores in the multiple regression analysis. The amount of formal reading instruction students received was negatively associated with reading proficiency. All factors combined accounted for 72% of the variation of PIRLS reading scores, with is very high (table 5).

Table 5: Multiple Regression: Complex (Full) Analysis

Predictors	beta	t	p
SES home	-0.02	0.122	0.9
Literacy	0.164	1.343	0.19
Library	0.493	4.801	0
Instruction	-0.483	3.454	0.002

 r2 = .72

The simple analysis

In the simple analysis, one predictor was chosen to represent each factor, one that was felt to be most representative of the factor we were interested in investigating.  For SES/Home, only one measure of socio-economic status was used, the Human Development Index (HDI) developed by the United Nations. The measure of literacy used was SSR (sustained silent reading), the percentage of students who read independently in school every day or almost every day in each country. The library factor was represented by the percentage of school libraries in each country with over 500 books.  Instruction was represented by the average hours per week devoted to reading instruction in each country. Inter-correlations among these variables are in the Appendix, table A7).

Table 6: Multiple Regression: Simple analysis

Predictor	beta	t	p
SES home	-0.41	2.74	0.005
Literacy	0.161	1.343	0.143
Library	0.346	2.75	0.005
Instruction	-0.186	1.4	0.085

 r2 = .63

The results are quite similar to the complex solution, except that SES, as measured by the HDI, is now a significant predictor (table 6).

Conclusion

 In all of the multivariate studies considered here the library emerges as a consistent predictor of reading scores.  This is remarkable, especially when we consider that the measures used are crude: library holdings, and even general circulation, in the case of public libraries.

Of course, providing access is only the first step: Even with access, some children (but surprisingly few) will not read.  The research literature consistently indicates that rewards for reading are not effective (McQuillan, 1997; Krashen, 2003; 2004), but that read-alouds and conferencing do help.  But in order for these approaches to work, the books need to be there.

But what is clear is that libraries definitely matter and they matter a lot.

Inspection of the betas in the tables reveals that access to books in some cases had a larger impact on reading achievement test scores than poverty (tables 1,3, 4), and in other cases had nearly as strong an impact (tables 2,5).  This suggests that providing more access to books can mitigate the effect of poverty on reading achievement, a conclusion consistent with other recent results (Achterman, 2008; Evans, Kelley, Sikora, and Treiman, 2010; Schubert and Becker, 2010).  This result is of enormous practical importance:  Children of poverty typically have little access to books (Krashen, 2004). It seems that libraries can provide this access.

References

Achterman, D. 2008.  Haves, Halves, and Have-Nots: School Libraries and Student Achievement in California. PhD dissertation, University of North Texas. http://digital.library.unt.edu/permalink/meta-dc-9800:1 

Evans, M,  Kelley,  J. Sikora, J. and Treiman, D. 2010.  Family scholarly culture and educational success: Books and schooling in 27 nations. Research in Social Stratification and Mobility, 28(2), 171-197

Krashen, S. 2003. The (lack of) experimental evidence supporting the use of Accelerated Reader. Journal of Children’s Literature, 29(2), 9, 16-30.

Krashen, S. 2004. The Power of Reading. Portsmouth: Heinemann and Westport: Libraries Unlimited.

Krashen, S. & McQuillan, J. 2007. Late intervention. Educational Leadership,  65(2), 68-73.

Lance, K. 2004. The impact of school library media centers on academic achievement. In C. Kuhlthau (Ed.), School Library Media Annual. (pp. 188-197). Westport, CT: Libraries Unlimited.

Lee, J., Grigg, W. &  Donahue, P. 2007.  The nation’s report card: Reading 2007  (NCES 2007–496).  National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education, Washington, D.C

McQuillan, J. 1997. The effects of incentives on reading.   Reading Research and Instruction, 36, 111-125.

McQuillan, J. 1998. The Literacy Crisis: False Claims and Real Solutions. Portsmouth, NH: Heinemann Publishing Company.

Mullis, I, Martin, M., Kennedy, A. and Foy, P. 2007. PIRLS 2006 international report. Boston: International Study Center, Boston University.

Schubert, F. and Becker, R. 2010. Social inequality of reading literacy: A longitudinal analysis with cross-sectional data of PIRLS 2001and PISA 2000 utilizing the pair wise matching procedure. Research in Social Stratification and Mobility, 29, 109-133.

This blog is “backseat” in the sense that, like a backseat driver, I am for the most part not doing any of the “driving” of the research but merely commenting on the performance of the person behind the wheel.

My first real post is, however, an exception to the rule.  It’s a joint effort of original research written with my esteemed colleagues Stephen Krashen and Syying Lee about one of my favorite subjects, the importance of libraries.