Even Better Than You Think: More Good News for Incidental Vocabulary Acquisition

Share on FacebookShare on Google+Tweet about this on TwitterShare on LinkedInEmail this to someone

Study Reviewed

Frishkoff, G., K Collins-Thompson, C. Perfetti, and J. Callan. (2008).  Measuring incremental changes in word knowledge: Experimental validation and implications for learning and assessment.  Behavior Research Methods, 40(4), 907-925.

We have long known that we acquire most of our vocabulary, in both a first and second language, through reading (Krashen, 1989).  Our knowledge of new words comes both incrementally (little by little) and incidentally (as a by-product of our main activity, comprehension).  Various methods have been developed to measure that incremental growth, especially the partial knowledge that we may gain about a word, but that falls short of a full definition of the word or even an acceptable synonym.  Below the threshold of being able to translate (in the case of second languages) or give a dictionary definition of a word, it appears that there exists a wealth of other, previously undetected word knowledge that readers are acquiring.  The study I review here introduces a new method for capturing that vocabulary growth.

Subjects: U.S. university students (N = 21), all native speakers of English.

Treatment: Subjects were given 60 unknown words in English to read in sentences.  The words were very rare and unlikely to be known by the subjects. Each word was seen six times in different sentences, in either useful or “good” contexts or misleading or “bad” contexts.  A good context would help the reader determine the meaning of the word, such the word “abrogate” in this example:

This system has been weakened since 1983, and the current Liberal party government seeks to further weaken or abrogate it.

“Bad” context (so named by the researchers) were sentences in which the target word was used in a context that was appropriate for another, similar sounding and similarly spelled distractor word.  For example, the target word “abrogate” would be used in place of a distractor, “arrogate,” as a malapropism:

Traditional distributors…abrogate to themselves the role of determining what’s proper for their customers to read.

In a bad context, the target word is misused as a replacement for the distractor word, thus potentially misleading the reader into inferring an incorrect meaning.

Some of the target or tested words were presented in all “good” contexts, some with three good and three bad contexts, and some with one bad and five good contexts.  Accuracy scores were predicted to vary according to the “goodness” of the six contexts.

Measures:  Subjects were given a “synonym judgment test” before and after the experiment, in which they were provided the 60 target words and asked to select the best synonym for each.  Among the possible answers was a synonym for the distractor word, so the researchers could test just how far the bad contexts would lead readers astray. The synonym judgment test provided a pre/post measure of word knowledge gain due to the experimental phase of the study, in which, as noted, subjects actually read sentences containing the target words in good and/or bad contexts.

In addition, after each exposure to the target word in context, two additional measures of words knowledge were taken.  First, subjects also had to judge the semantic appropriateness of sample sentences that contained the target words. While not asked to give a definition, subjects had to understand enough of the word to determine if the test sentence using the target word made sense or not.

Second, subjects wrote down the meaning of the word (their best guess) after reading each sentence.  These responses were then analyzed using a recently developed method of statistical modeling called the “Markov estimation of semantic association” (MESA) to measure small, incremental growth in word knowledge after each exposure. Each guess the subjects gave as to the meaning of the target word was given a score indicating its “distance” from the correct definition. The MESA model used a variety of factors to determine distance, such stemming (words based on a common morphology), synonymy (words of similar meaning), co-occurrence (words that tend to appear together in the same context, such as “election” and “politics”), and associative strength (words one might give if doing a “free association” with the term), among others.

If the subject’s response had no possible links to a correct answer along any of the dimensions that were included in the model, knowledge was scored -1; a correct definition or synonym was scored 0.  This allowed the generated answers to be placed on a continuous scale marking their approaching accuracy to the word’s correct meaning, and thus revealing incremental growth that fell short of full knowledge of the word.

Here’s an example of how responses might be scored for the word abditive (meaning “hidden”) presented in all “good” contexts.  Remember that responses are more accurate as they approach zero on the MESA scale (taken from Example 1, p. 917):

TrialResponseMESA Score
1Avoidant-1.00
2Attitude-.83
3Sneaky-.50
4Secretive-.35
5Secretive-.35
6Hidden-.35

Results:  Words found in more useful contexts were, not surprisingly, acquired more quickly than those placed in “bad” or misleading contexts.  The average gain from pretest to post-test on the synonym test was about 12%, which means on average subjects picked up an accurate meaning of around seven of the 60 target words after six exposures.  This 12% figure appears to be for all context conditions combined, however, including the misleading ones.  The article’s Figure 2 (a) (p. 915) shows that when words are presented in all good contexts, the “pickup” rate was closer to 17%.  Subjects also were less likely to choose the distractor’s synonym when the word was presented in good contexts, showing another aspect of word knowledge growth.  Notice in the example above how each guess captures something of the meaning of the word, some aspect that shows the reader closing in on the real definition.

The MESA measure confirmed that words presented in good contexts were more easily acquired than those in bad contexts. More importantly, the MESA measure showed an incremental, linear pattern of more accurate word knowledge after each “good” context exposure, indicating that each guess was getting nearer and nearer to the correct definition.

Comments:

1. The important thing about this study is the methodology for detecting small changes in word knowledge after each exposure.  Frishkoff et al. have shown that even when subjects are asked to produce a meaning of target words encountered incidentally (versus merely to recognize the correct meaning in a multiple choice format), there is evidence of considerable cumulative growth in knowledge, knowledge that often falls short of the “correct” answer.

The measurement of the acquisition of partial word knowledge is by no means a new thing. The MESA methodology is merely a more sophisticated approach to the problem, and a reminder that many current measures will likely miss increases in vocabulary growth that “fly under the radar” of a translation or even a receptive measure such as a multiple-choice test, and thereby underestimate the incidental growth in word knowledge through reading.

2. The study’s finding that subjects picked up 12-17% of words presented in good or mostly good contexts is in line with the rates of “pickup” found in studies of incidental acquisition in an L2 setting. But this figure is, as already indicated, only part of the story.  The true amount of new knowledge gained incidentally of unknown words is much greater, as the MESA scores demonstrate. In the case of the Frishkoff et al. subjects, something was probably being picked about several of the words that appeared in good or mostly good contexts, making the real gain from reading much more than the 12-17% that passed the threshold of being “known” on the post-test.

The MESA measure is an interesting advance in assessing incidental acquisition, helping us detect the previously unseen incremental growth that takes place even after a single pass or two at a word. This also means that current approaches used in vocabulary acquisition research likely underestimate the effect exposure has on word acquisition.

The case for incidental acquisition through reading keeps getting stronger and stronger.

Works Cited

Krashen, S. (1989). We acquire vocabulary and spelling by reading: Additional evidence for the Input HypothesisModern Language Journal, 73(4), 440-464.

Share on FacebookShare on Google+Tweet about this on TwitterShare on LinkedInEmail this to someone