I recently did another YouTube interview (to be posted shortly) and realized that I had forgotten to post this one (with James Stubbs) from last year.
We talked about the research on the use of flashcards in programs such as Anki to building second language vocabulary. Transcripts (cleaned up a bit from YouTube) are below for those not wanting to watch the videos.
Note: I’ve added a correction below to one of my calculations discussed in the first video.
The interview was published last year in two parts.
Transcript – Part 1
James:
Hello, and welcome back to Future Multilingual. We have another one of our episodes with a language acquisition expert, Dr. McQuillan. Good afternoon, Dr. McQuillan, lovely to see you.
Jeff:
Thank you.
James:
Recently, we’ve been talking quite a lot about language hacking and this common opinion that you can quickly memorize yourself into a language in a decontextualized way. You know me — I think language is about getting closer to a culture and implicitly acquiring language while you do that. In order to make this point a little stronger, I have invited Dr. McQuillan back, who is going to talk us through the process of language acquisition. Well, it doesn’t mean memorizing words.
Jeff:
I’m going to start off, James, if it’s okay with you, by going through a few terms that people might see either on websites or when they do research on this topic. Googling “what is the best way” will bring up some terminology, especially that we use in the research world, that will be useful. I want to take a deep dive into what the research about acquiring language is really all about.
There’s this view that Steve Krashen is most associated with — the view that language can be acquired, meaning your subconscious or unconscious knowledge, and that language can be consciously learned. A lot of people, including many researchers, have argued that the quickest way to, for example, improve your vocabulary is to simply memorize words. This has a long tradition in experimental psychology going back to the late 19th century. In fact, in many ways, experimental psychology was founded upon some studies related to what is called “verbal learning,” which is memorizing words.
Typically, these studies are set up in something called “paired-associate learning.” Paired-associate learning is basically our old friend, the flash card. You have one piece of information here and another piece of information there, and you’re trying to associate or memorize those two pieces of information. It could be a word and the definition in your own language, or it could be the word and then the translation of that word in another language. Now, of course, websites like Anki and programs like Duolingo have this flash-card mentality built into their systems. In Anki, that’s really all it does.
Sometimes paired-associate learning is presented as a list. You have a list of words here and a list of translations there, and you give them to someone who tries to memorize the two. One of the things that was discovered early on and one of the ongoing issues is whether it’s best to memorize words all at once — what’s called “mass learning” — or if it’s better to space them out over days, which is called “spaced repetition.” So, I study the words on Monday, then again on Wednesday, and then again on Saturday. Tons of research has been done about which is the best schedule to use.
Now, all of this presupposes that you can actually memorize words consciously, and that’s the way we get our vocabulary. That’s the learning view. There’s another view that says the size of our vocabulary is so large that conscious learning — deliberate learning, rote memorization — can’t possibly account for all of it. These terms basically all mean the same thing. Instead, we acquire vocabulary not by trying to memorize each word. We don’t make 20,000 trips to the dictionary to get a 20,000-word vocabulary. Instead, we acquire words incidentally.
That means we’re not trying to memorize words when we’re reading something. We see a word in context, figure out a little bit of that word, and it enters into our brain unconsciously, into our acquired system. This is called incidental acquisition, or simply acquisition. We often say that acquisition of vocabulary is incidental, meaning you don’t try to do it. It comes from natural language comprehension when you’re involved in listening or reading. It’s also incremental, meaning you typically pick up a little bit of the meaning each time. Maybe it’s 10%, maybe 15%. I’m using quantifiable terms for something that isn’t easily quantifiable, but the idea is that each time you see a word, you’re picking up a little bit more of that word’s meaning.
You do that all without even thinking about it. Suddenly, your vocabulary increases, and you won’t even notice it. It just starts happening.
James:
I have a question. I saw the term “semantic geography” the other day, and I thought it fit well with this idea. Doesn’t all new learning connect to some sort of old learning? If you’re incrementally building this into a semantic geography, incrementally connecting it to ideas you previously had, this would make sense. Is that a way of understanding it?
Jeff:
Absolutely. This comes from the cognitive revolution of the 50s and 60s, where we started talking about terms like schema, schemata, and networks where one word associates with another. When you come across a new word, it enters into your semantic geography or web.
James:
It might be different for each of us. For example, if my music teacher was also a cow herder, there would be a big connection between the words “music” and “cows” for me, but not for you.
Jeff:
Exactly. It’s a personal thing; it depends on your experience, just like all language acquisition.
Now, the debate has been going on for many years about whether rote memorization, in general, is an effective way of increasing knowledge on a certain topic. I don’t want to talk about other areas; I want to focus on vocabulary because it’s the one that has been studied the most. There are hundreds of studies on this topic over the last 120-130 years. Second language researchers like Norbert Schmitt and Paul Nation have claimed that you can acquire words by reading them incidentally and unconsciously.
James:
But to deny that would be an extreme position, wouldn’t it?
Jeff:
Yes, some might, but everybody acknowledges that there’s subconscious or unconscious acquisition. The problem arises when additional claims are made by people like Norbert Schmitt, who say that learning vocabulary through rote memorization is much faster. The standard position in the field is that you can pick up words by reading and listening, but it’s slow. If you want to make progress quickly, you should start memorizing words using paired-associate learning or spaced repetition.
Let’s talk about some of the studies done on this topic. A typical study in this area involves taking a group of students and giving them a list of 15, 20, or maybe 30 words to memorize. Sometimes this is done in class; sometimes they use a computer program like Anki. They’re asked to memorize a certain number of words.
James:
That’s not very many words.
Jeff:
Exactly, and that’s an important point. These studies usually focus on small numbers of words because researchers aren’t trying to show that you can memorize thousands of words. They’re trying to demonstrate a model about how the brain works, how we learn words. They’re studying certain aspects of that to prove whether their model is correct. Their purpose is not to show that Anki works; their purpose is to show something about cognition and memory.
James:
Which is obviously more interesting.
Jeff:
Right. The problem comes when we try to translate that research into the real world. Take, for example, one of Nakata’s studies. He took a group of students and gave them 16 words to memorize using spaced repetition in class. They ended up knowing about 85-90% of the words. If you look at how long it took them to memorize those words, it looks like they memorized 25-30 words an hour, extrapolating from their data. But they only memorized 16 words.
James:
So you’re extrapolating from that small sample.
Jeff:
Exactly. Some experimental psychologists say that whether it’s 16 words or 1,600 words, the same principle applies. But very few studies have actually tested this. Only three studies that I’ve found directly relate to real-world application.
One of the first major studies was done in 1908 by William Thorndike, one of the godfathers of modern psychology. Thorndike took a group of adults and asked them to memorize 1,200 words in German. All of them knew a little bit of German, but they weren’t fluent. He gave them a list of 120 words and asked them to memorize them.
“Keep track of how long it takes you to do it, and I’ll come back when you think you’ve gone through all the lists. We’ll come back, and we’ll test you.”
So, he comes back, and he tests them, and he finds out that they knew about eighty percent of the words. So, they knew more than a thousand words — very impressive. And it took them about, they were learning, I want to get the right figure, about 30 words an hour, is what they were able to memorize.
James:
Okay.
Jeff:
Right, which sounds really impressive.
James:
It does.
Jeff:
But now I have to introduce another important distinction: immediate post-test and delayed post-test. Immediate post-test is the test you give somebody right after you do something. So here, kids memorize these words, and then five minutes later, you give them a vocabulary test. That’s an immediate post-test, which is basically like no long-term vocabulary.
James:
But this is what most people do for exams all over the world, no?
Jeff:
Yes, all learning is measured by immediate post-test. But the immediate post-test, of course, is not of interest to us, right?
James:
No, not if you want to speak a language long-term.
Jeff:
No, you’re not going to memorize and then use those words in the next five minutes.
James:
Exactly.
Jeff:
You want words that you’re going to know next week or two weeks from now. So, the important measure is not the immediate post-test; it’s the delayed post-test. Now, there’s no standard amount of delay in these tests. Sometimes it’s a week, sometimes it’s two weeks, sometimes it’s a month. Thorndike came back to his people, I think about five weeks later, and he tested them again. And guess what? They lost half of their vocabulary. They only knew about 500 odd words, which, if you take the amount of time that they spent — I think they spent about 35 hours memorizing these 1,200 words — that works out to be about 16 words an hour.
James:
Right.
Jeff:
Now, notice that 16 words an hour is nowhere near the claims that you will sometimes see researchers make. I mean, one researcher says, “Oh, you can learn up to 100 words an hour.” No, you can’t. There’s no evidence you can.
James:
Yeah.
Jeff:
Thirty-five words an hour? If you look at an actual study that tried to do what is being advocated, they were getting half of that efficiency. So, it turns out to be about 16 words an hour. And that’s the interesting thing: the closer you get to reality, the smaller these numbers become.
James:
I have a question here.
Jeff:
Sure.
James:
It depends on how you set the study up, but is there interesting research about whether nouns are easier to memorize than verbs? Instinctively, I would assume nouns are easier.
Jeff:
Yes, there is. Well, the term that’s used in the research is often “imaginability” or “concreteness.” So, the more concrete the word that you’re trying to match, like “bicycle,” for example, or even “running” — although it’s a verb, it’s easy to imagine somebody running. But words like “when” or “seen” or “appear,” those are very difficult words to memorize. And what you’ll see is a huge difference. Now, here’s the interesting thing — I was going to talk about this later, but since you brought it up — if you look at, say, the 300 or 200 most commonly used words in any language, you can just go to Wikipedia and look this up.
James:
Just to interrupt your smooth flow.
Jeff:
That’s good. Yeah, this is a tactic advocated by productivity gurus all over the world.
James:
Okay.
Jeff:
All right, here’s the problem with that. If you actually look at that list of words, what you’ll discover is that there are hardly any concrete or imaginable words on that list. Half of the words in English, for example, are made up of very abstract or non-imaginable, non-concrete types of words. So, you know, words like “the,” “and,” “if,” “then,” and conjunctions. These are words that you’re not going to be able to imagine. And those are ironically the words you would think you’d want to start with.
James:
Right.
Jeff:
Because they’re grammatically so complicated to try and memorize.
James:
Like “ser” and “estar” in Spanish.
Jeff:
Exactly.
James:
Yeah, absolutely.
Jeff:
And that’s — we could talk about grammatical or syntactical acquisition as well, and there the difficulties are even greater. Also, many of the most common words have multiple meanings.
James:
Right.
Jeff:
So, which word do you memorize for that particular concept? There are all sorts of practical problems. But that’s why I want to talk about some of these studies, James, because it’s easy to talk about the theoretical aspect of it, but if you get down to the nitty-gritty, does it actually work? Let me give you another example of a study like this. This one is quite interesting because it complements the Thorndike study that I mentioned earlier, the 1908 study, very well.
There were a group of, I think, a husband, wife, and their children, all of whom were professional psychologists. I might be wrong about the relationship, but they all had the last name Bahrick. Harry Bahrick was the lead author of one of my favorite studies. This is a study that took nine years to complete.
James:
Wow, nine years?
Jeff:
Yes. Think about that. That’s an experimental psychology study that took nine years to complete. The study was published in 1993, and it’s often called the Bahrick family study.
James:
Okay.
Jeff:
Here’s what they did. They said, “We’re going to try to memorize 300 words of a language that we’re not fluent in.” So, I think for three of them, it was German, and for one of them, it was French. And to make this as realistic as possible, they aimed to memorize them over a long period of time. In one case, they did it for six months. They took the 300-word list and broke it up into smaller lists of 50. So, they tried to do one every 14 days, reviewing their list for a six-month period. The idea was to get up to a very high standard, say, knowing 90% of the words.
What they did was test different spaced repetition schedules. One was 28 days, another was 56 days, etc. Then they went back one year, two years, three years, four years, five years later.
James:
Okay.
Jeff:
Now, here’s the interesting thing. With no extra work, only the initial input —
James:
No additional study?
Jeff:
No, that’s a good question. They studied them initially, and then they went back two weeks later. For the 14-day spacing, if they didn’t know a word, they studied it again. So, they were testing and studying each time they did these repetitions. It took them — my estimates, and I published an article about this in 2016, not only on this topic but including this calculation — I estimated that, if you take the amount of time it took them to learn these words to a high level, they ended up learning about seven words an hour.
Not 35 words an hour like Nakata, not a hundred words an hour like you see on the internet sometimes. In other words, if you try to do this in the real world and aim for long-term retention, you’re not going to see those sorts of efficiencies. You’re going to see something closer to seven to ten words an hour, not 35, not 50, not 100.
James:
So, if I just go back to what I know — I mean, I’m hard on maths — but you do see that qualification that sort of says A0 to A1 is 90 hours. No, do you know what I mean?
Jeff:
A1 to A2, yeah.
James:
Yeah, that’s not used so much in the US, but I know the European framework.
Jeff:
So, seven words an hour in nine hours, you’ve got 70 words.
James:
Yeah, you must need like, so times nine, that’s… well, see, that’s where I’ve exceeded my abilities. But what I’m doing is nowhere near — at 630 — nowhere near making the steps. Do you see what I mean?
Jeff:
Absolutely, I mean, yeah, close to a thousand words to be making a step from one level to the next, no?
James:
Yeah, absolutely.
Jeff:
Now, to be fair, the memorization people would say, “Oh no, that figure is way too low. Seven’s too low.” To which I say, “Okay, about 16 words an hour.” Even 16 words an hour doesn’t seem like all that much, right? But let’s talk about incidental acquisition because this is where things get a little tricky.
If you look at these researchers — now, they’re not dumb, they can do the math just as well as you and I can — they say, “Well, okay, sure, but incidental acquisition is even slower.”
For example, there was a study — I think he’s Dutch, his name is Jan Hulstijn. I think that’s his name; I’m probably mispronouncing it, and my apologies, I don’t know any Dutch. Jan Hulstijn did a study where he took a group of students and gave them a passage to read. I think it was about a thousand words long, and into that passage, he inserted words that he guessed the students didn’t know. I think there were 12 words he tested.
James:
In fact, that seems to be something that’s perfectly possible to do.
Jeff:
Yeah, I’m not sure about Hulstijn’s study, but what’s often done in these studies is, to make sure students really don’t know the words, they make one up. They use what’s called a “pseudo-word.” A pseudo-word is basically a nonsense word that sounds like the language. It sounds like English, so a word that sounds like it could be real, but it isn’t. They know the students will never have seen the word before.
They go through, and after the students read the passage — this is called the “read and test” study — they read the passage, and then they don’t tell them they’re going to give them a vocabulary test. That’s very important. They say, “Okay, read this passage,” and then when they’re done, they say, “We’re going to give you a comprehension test,” but they don’t give them a comprehension test. Maybe they do, but that’s not what they really care about. They say, “Now, we’re going to give you a vocabulary test, and we’re going to see how many of those 12 words you picked up.”
What you find in those studies is that the number of words people can actually acquire by seeing them once or twice — which of course is not very much — is very low. In Jan Hulstijn’s study, it was like 0.9 words averaged out across the whole group. That basically means they only learned about one of those 12 words in the six and a half minutes it took them to read that particular passage.
James:
Okay.
Jeff:
Now, what Jan Hulstijn and people like Norbert Schmitt and other researchers in second language will sometimes point out to you is, “Well, look, one word. That’s pathetic. In six minutes, I could memorize five words or ten words.” There’s a problem, however, with this logic, which I want to get to in a second. But on the surface, it seems as though rote memorization, 25-30 words a minute, versus incidental acquisition, one or two words… yeah, it seems like they’ve got you, right?
James:
Okay.
Jeff:
Let’s now look at that a little closer and find out why that’s absolutely wrong, why they’re looking at this from the completely wrong angle.
Here’s what you have to understand. When we’re looking at incidental acquisition, let’s say I have — let’s go back to that passage of a thousand words. What is typical for a second language learner, especially beginning or intermediate language learners, is that in order to understand a text with good comprehension, we usually find that they need to know between 95 and 98 percent of the words on the page.
That doesn’t mean you can’t understand something if you only know 90 percent of the words or even 85 percent, but your comprehension will improve the more words you know, generally speaking. Which is why it’s so much easier to go from one European language to another, no? Because there are so many cognates.
James:
Knowing 90% is huge. Yeah, it helps a great deal.
Jeff:
Exactly, exactly. So, here’s what these studies will usually do. They’ll give students a passage, second language students — let’s say we take a group of college students who are learning English — and we give them a passage that’s a thousand words, a reading passage, a text. We know they’re going to know 95 percent of the words, but there’ll be 5 percent of the words they don’t know. So, there are 50 words they don’t know.
James:
Right.
Jeff:
The researchers, because they want to test how good they are at picking up words incidentally, let’s say they stick in 10 words. So, they’ll test them on 10 words, okay? Let’s say they find out they only know two of those 10 words after they’re done reading the passage. They look at that and go, “Oh, it’s only two. They only picked up two of the ten words, 20 percent.”
What they don’t seem to understand is that they’ve picked up 20 percent of all the unknown words in the passage.
So they didn’t pick up two words; they picked up 20 percent of 50 words!
James:
Okay.
Jeff:
What they’re forgetting is that you have to account for all the unknown words in the passage, not just the words that you’re testing on.
This is the old joke about the drunk looking for his keys under the streetlight because that’s where the light is. You can’t just look where the light is. You can’t just look at the words you tested. You have to look at the percent that the reader acquired and then apply that to the rest of the population of unknown words. It’s a matter of confusing sample with population.
One of the leading researchers in second language vocabulary — I won’t say her name — but she made this really fundamental mistake in one of her articles. I had to publish a response and say, “Actually, no, that’s wrong. You’ve got to look at the full percentage.”
So, let’s go back now and do a little more math. Stuart Webb, who was a student of one of the great researchers in vocabulary of the 20th century, Paul Nation — well, still the 21st century — Paul Nation is interesting. He’s at the University of Victoria. His views are — he’s sort of, uh, got a foot on both sides. He thinks incidental acquisition is really important, but he also thinks that you should try to memorize. I love Paul, he’s a brilliant guy. I disagree with him at my peril, but I do think he’s wrong about rote memorization. And here’s why.
Let’s go back and do some math. How fast, how many words can you read in a minute if you’re a second language learner? Well, it depends upon your level of fluency, but if you’re, say, a lower intermediate to intermediate, you’re probably reading at about 150 words a minute. 120 to 150 words a minute, assuming you’re reading something where you know, what we said earlier — remember, I talked about knowing 95% of the words?
James:
Yes, we talked about that as vocabulary coverage.
Jeff:
So, if you have vocabulary coverage of 95% to 98% of the words — you know 95% to 98% of the words — you’re probably reading around 125 to 175 words an hour. Let’s call it 150 words an hour. Now, that’s much slower than you’d probably read in your native language. Most native speakers of English, for example, are reading between 200 and 300 words an hour.
James:
Okay.
Jeff:
Depending on what you’re reading and the purpose. I mean, some things you read really quickly because you don’t need to get everything. So, it depends on your purpose for reading. But typically, for comprehension, we talk about two to three hundred for your native language and about 150 for your second language.
Now, let’s say you’re reading a text that has 95% of the words that you know. So, you can read it with some comfort and good comprehension, but there’s still going to be 5% of the words you don’t know. Well, if 5% of the words are unknown in a text, that means you’re coming across about seven and a half words every minute that you don’t know. If you’re reading 150 words a minute, and you don’t know 5% of the words, that’s about seven and a half words every minute, which is about, say, 450 words an hour. Something like that.
If you’re only picking up, let’s say, 15% of those words — and I take that figure, 15%, from a recent meta-analysis done by Stuart Webb, I mentioned his name — Stuart Webb was a student of Paul Nation. He did a meta-analysis. A meta-analysis is when you take a bunch of studies and summarize the results of a whole bunch of studies. He looked at a whole bunch of studies. It’s not a perfect paper, but I’ll use it as a basis because it’s recent and he’s an advocate of teaching by rote memorization. So, I’m not cherry-picking here. He came up with a figure of about 15% for delayed post-tests. In other words, if you test someone two or three weeks later, on average, they will have picked up about 15% of the unknown words in a passage.
If you multiply that 15% by that seven and a half unknown words, you get, let’s say, 450 words an hour. If you multiply the pickup rate — let’s be a little more conservative and call it 12% because that’s actually the figure from Stuart Webb, not 15%.
If you take the 12% of words that you’re picking up and multiply by the number of words in an hour, if you’re reading a text with 5% unknown words, that’s 54 words an hour. [SEE NOTE BELOW]
James:
Okay.
Jeff:
So, let’s be more conservative. Let’s say you know 98% of the words, and there are only 2% of the words that you don’t know. That ends up being about three words every minute if you’re reading at 150 words a minute. If you pick up 12% of those words, that would be around 180 unknown words an hour. Twelve percent of 180 is 27 words.
James:
And obviously, if there are fewer unknown words, you would naturally increase your reading speed.
Jeff:
Absolutely, that’s right. And you would also probably increase the likelihood of that percentage going up because the more unknown words there are in the passage, the harder it is to comprehend the passage and figure out unknown words in the passage.
James:
It’s like a lever, right? The closer you are towards the end, the more power you have because you’re incrementally getting them until you hit that subconscious eureka moment where your mind says, “I know what that means.”
Jeff:
Exactly, exactly. So, the math here does not work out in favor of rote memorization, to summarize. If these figures are correct, we’re talking about somewhere between 25 and 50 words an hour.
James:
Okay.
Jeff:
What do you get from rote memorization? Well, they say you get somewhere between 25 and 35, maybe 50 words an hour. In other words, what seems like a huge advantage is, in fact, just a slight advantage.
James:
Yeah, incidental acquisition. I mean, and there are other factors here. We brought Alfie Kohn into the discussion about motivation, I’m sure.
Jeff:
Yes, even setting aside the emotional and psychological impact of trying to do that level of memorization, the math doesn’t work out. Even if you could show an advantage to rote memorization — because I’m sure people will say I’ve overestimated here, or underestimated there — even if there’s a numerical advantage, there are still several other good reasons why rote memorization is a terrible strategy for someone trying to improve their vocabulary.
James:
Because — and we’re going to stop this recording in a moment and move to another video — I want spontaneous access to this vocabulary. I don’t want to be searching for it in my mind. It seems to me, anything I’ve ever memorized — because I did study law when I was younger, which is an exercise in memorization for anyone interested, memorizing cases — I never had instant access to those cases. I wasn’t using them implicitly, spontaneously, which seems to be massive.
Jeff:
Yes, absolutely. What you can do with what you’ve memorized is the next shoe to drop here.
James:
Exactly.
Jeff:
So, we’re going to take a break for a moment. Pause there. I’d recommend reflecting on what you’ve just heard and trying to apply it to your own learning because I think that’s the best way to understand it. We’ll see you in a moment.
NOTE:
In the interview, I stated that if you read 150 wpm for one hour (9,000 words) with a 5% vocabulary coverage, you will encounter 450 unknown “words.” This is correct if we use “words” (as we do in normal conversation) to mean what linguists call tokens. The sentence “She buys bread and she buys milk” has seven tokens (number of words in the sentence). But two words are repeated (she, buys), so there are actually only five unique word families.
When we’re calculating “words” acquired, we typically look at the word families, not tokens. It is precisely because members of a word family are repeated that we are able to acquire new ones from a text.
Thus, while you will see 450 unknown tokens in those 9,000 words of text, it is likely that some of them will be repeated at least once, and hence the number of word families will be smaller.
How much smaller? That will vary by the text. Some very rough calculations I just did (I’ll do more careful ones and post on it in the near future) on a 15,000 token passage from the teen novel Twilight found that there were 488 word families appearing in the unknown 5% of the text, with 708 tokens. If we use that ratio and adjust for 9,000 words, there would be about 288 unique word families that you would encounter for every hour of reading. Let’s call it a range of 250-300 words.
So you would acquire 30- 36 words per hour rather than my original estimate of 54 words.
Note this is still much higher than what we see in the Bahrick family study (about seven per hour) and in the Thorndike study (around 16-17 per hour). The math works out in favor of flashcards only if you extrapolate from studies such as Nakata’s, which use a very small number of words and are (I would argue) not reflective of what is typically advocated for flashcard use.
Given the huge opportunity costs of using flashcards vs. getting good comprehensible input (discussed in the second video), CI is still the clear winner.
Transcript – Part 2
James:
Welcome back to my interview with Dr. McQuillan. We left you talking about efficiency and time spent on memorizing and on input — reading or listening. Well, it was mainly reading. Welcome back, Dr. McQuillan.
Jeff:
Thank you, James. Great to be here.
James:
We’re going to pick it up by talking about the type of knowledge because, to me, this is key. The different approaches to language learning leave you… So, take it away.
Jeff:
Okay. Well, last time, as you said, we talked about whether memorization was actually any more efficient. I would argue that in many ways it’s not. However, even if it were, there are some serious limitations to the kind of knowledge that you get when you try to memorize something through rote methods, such as Anki Web or one of the other electronic flashcard methods.
And that is this: you would think that knowing a word is knowing a word. That if you know a word and you know the definition, that’s it — that’s all you need to know. Turns out it’s not that simple. It turns out that certain kinds of knowledge acquired through things like rote memorization don’t seem to have the same effect on us. They don’t seem to, in fact, be available for use — if you will — online, which is a term that psychologists like to use nowadays. That learning doesn’t seem to be available for you to use, for example, to understand a conversation or to read a passage that has that particular word in it.
James:
Or to just speak spontaneously.
Jeff:
Exactly. Now, this has actually been known in the reading world for about 60 years. It was in the late 60s, early 70s, when experimental psychologists got interested in whether the classic, standard way of trying to teach kids new vocabulary actually made any difference in terms of their reading comprehension.
Because, remember, the purpose of learning a word or acquiring a word is not so that you can pass a vocabulary test, right? It’s so that you can be able to use it.
James:
Have you seen the one about the Scrabble player who memorized all the words in French? He was able to play Scrabble, but he can’t speak a word of the language.
Jeff:
Yes, absolutely.
James:
Now, people try to say, “Oh, that’s because he doesn’t know the grammar,” but it’s not that, is it? It’s much more to do with how he knows those words, isn’t it?
Jeff:
Absolutely, absolutely. This is such a fundamental point that we’ve known in first language reading for, as I say, 50 to 60 years. But it’s been completely forgotten or ignored by people in the second language world, and I don’t know why.
Here’s what we know: if you sit down and take a group of kids, and you say, “Okay, kids, we’re going to have you memorize these ten words,” and then you give them flashcards and test them. You make sure that at the end of this little five- to ten-minute session, they know the words, right? The kids know the words, no problem.
Then, you give them a reading passage, a text, and you say, “Okay, now read this, and then we’re going to ask you some questions about it.” They read it, you ask them some questions about it, and then you take another group that didn’t get any of the instruction. You give them the same passage, they read it, and the two groups do the same.
The vocabulary instruction didn’t do any good, in other words, in terms of actually comprehending the passage. What this told them is that these kinds of — what they later termed — “shallow instruction.”
James:
I like that term — shallow instruction.
Jeff:
Yes, they know the word, but they don’t know it at the acquired level — the level you would need to actually use it in comprehension. Now, it’s okay if you just give them a single sentence. So, if you teach a kid a word and a definition, and then you give them one sentence, and the sentence has the word in it, they’ll understand the sentence. But if you put it in a reading passage, it doesn’t work.
James:
That seems to make a lot of sense. I think I asked you this last time, I can’t remember, but did you see any of the pedagogy lab stuff from Columbia University in New York? One woman looks at readers, and they looked at adult learners who learned to read but can forget the skill of reading.
Jeff:
Yes, that’s because they said, “Look, if you’re not a fluent reader and you learn as an adult, you can lose that skill.”
James:
Which, to me, suggests we’re doing these things in completely different parts of our brain.
Jeff:
Absolutely.
James:
So, if you’re memorizing a word, you could have that in one part of your brain, which allows you to play Scrabble, for example, but it doesn’t allow you to process it as part of a long stream of ideas.
Jeff:
Yeah, I’m not familiar with that, but I’ll look it up later. That sounds really interesting.
So, to continue on, we know that this shallow instruction only works for things like single, isolated sentences. It doesn’t work for reading passages. The theory that we have, and my explanation of that, is precisely as you say — acquired knowledge of vocabulary is different than rote memorization, conscious learning, or deliberate knowledge of vocabulary.
This approach of doing shallow vocabulary instruction was then considered a waste of time. What happened is teachers and researchers started looking at other ways. This is a great example of the problem when you have researchers doing this kind of stuff. Instead of saying, “Does shallow vocabulary instruction work? Maybe we should just let kids do a lot of reading,” they said, “Oh no, we need to give them even more vocabulary instruction. We just need to change the kind of vocabulary instruction we give them.” So, we went from shallow vocabulary instruction to what was called deep or rich vocabulary instruction.
Instead of just giving kids words and definitions, or words and translations if it were second language, we would give them lots of activities. We would contextualize it. Now, that’s better. No question that is a better approach. However, it’s still highly inefficient. I’ve just written a series of articles over the last four years to show that. But to keep on the point here, the point is that when you try to memorize words consciously, you’re not going to be able to use them, at least according to the research we have so far, in the same way.
They replicated these studies in second language. Patricia Johnson did a study in 1982. Tom Hudson did a study in 1981, published in major second language research journals, which showed the exact same thing going on in second language. If you teach somebody the words — and mind you, they’re teaching them the words right before giving them the reading passage — which is, again, not real life. In real life, you learn the word and then come across that word two weeks later in a passage. They were giving them the passage immediately after, and even then, they weren’t able to use that knowledge to comprehend the passage any better than somebody who hadn’t been taught the words at all.
James:
That’s a huge disadvantage.
Jeff:
Yes, a huge disadvantage. Now, there are some people in the last 10 to 15 years who have tried to come at this in a different way. A researcher by the name of Irina Elgort, who was also a student of the great Paul Nation at the University of Victoria — anyway, he’s in New Zealand somewhere.
Irina Elgort did some studies in which she tried to show that, actually, if you memorize words with vocabulary and you give someone a certain kind of specialized laboratory test that deals with things like reaction time — what they do is they flash a word up, and they see how fast it takes you to decide whether it’s a word or not a word. This is a sort of classic experimental psychology method. If you do that kind of study, it looks as though, at least in some but not all of their tests, that the rote memorization word was about or close in the neighborhood to words that someone had normally acquired.
The problem with her research is that she doesn’t take the next step and say, “Okay, let’s now give them a passage with those words in it and see if they can understand it.” In fact, she doesn’t even cite the previous research. Now, again, I don’t want to be too harsh on Elgort because her purpose in doing her studies was to demonstrate certain theories about memory and learning. Now, I’m not going to argue whether those theories about implicit learning and explicit learning are correct or incorrect. All I’m saying is that her study doesn’t actually show that you can memorize words and then use them, as she says, “online,” meaning in real-time production or comprehension.
James:
I mean, to me, there’s a big difference, even like recognizing them in a text.
Jeff:
Well, you know, my passive vocabulary in Spanish is huge. There’s a huge amount of words that I can recognize in a text.
James:
Am I using those? I mean, we’re still a long way away. I mean, I know that we generally speak quite simply, but I’m not even sure I could produce them in a piece of writing.
Jeff:
Well, that introduces another distinction between productive and receptive vocabulary. But even without going into that and just talking about the receptive level, Elgort’s studies don’t show this because she hasn’t actually done the test. Now, what she would say is that reading involves all sorts of things — more than just knowing a word. If we did that kind of test, there would be all sorts of confounding variables. So, she wanted to keep it simple and just look at her one little thing. To which I say, “Great, fine, if you want to look at your one little thing, that’s fine, but you can’t have your cake and eat it too.” You can’t say, “I can only look at this one thing because otherwise, it would get all muddled,” and then, in the next paragraph, say, “Oh, vocabulary is definitely a good strategy for people because I’ve shown it in my study.” No, you haven’t shown it in your study. You’ve only shown that you’ve been able to get similar results on this particular test, and we have no idea how that translates into the real world.
James:
So, just to open it up to what people’s experiences potentially are, I mean, if people are teachers or if people read a lot in a second language, confounding variables are everything. Like, for example, “I didn’t know that word could be used in that way. I didn’t know you used it in that context.” When you get words suddenly, my mind has gone blank, but you get direct translations in English and Spanish, and people don’t realize that it covers several different meanings, right?
Jeff:
Yes, absolutely.
James:
And that is something that people really struggle with. So, all of those things you’re only going to get by incrementally acquiring stuff, right?
Jeff:
Absolutely, absolutely. It’s going to take time. Now, to her credit, it’s a weird thing. If you read Elgort’s studies, she definitely is aware that she hasn’t really shown the case. But like most researchers, she just can’t help herself. She’s got to jump to the implication section and say, “Well, what does this mean in the real world?” Then she’ll say, “It means that memorization of vocabulary is useful.” And I’m just saying, “You know what? You haven’t shown that.” We have half a dozen studies that show that doesn’t happen, right? We’ve already established in the 70s and 80s that this doesn’t happen, so why should we think that your reaction time study is somehow indicative of the ability to use those words in comprehension or production? It isn’t. And until she can show that it is, it’s not evidence. The current knowledge that we have is that you can’t use it in the same way as if you were to acquire it, say, incidentally, through reading and listening.
James:
Yeah, that makes sense.
Jeff:
Now, I want to say two more things if I can, James. I know we’ve gone on for a long time. There are two other reasons why I think rote instruction is a very bad idea in terms of a general strategy. One of them is what we call, as you know, the term “opportunity costs.” There are opportunity costs, which is a term in economics, which is basically, “Well, if I’m going to do A, that means I can’t do B,” right? We live in a world of choices. So, if you decide you’re going to study 10 hours on your Anki vocabulary list, that’s 10 hours that you’re not going to be reading or listening or engaging with the culture or picking up newspapers or anything else that you might benefit from.
Now, people who are language hackers and others will say, “Well, look at all these words that we’re learning during these 10 hours.” Yeah, well, we’ve already established that that efficiency calculation is probably at least slightly bogus, if not completely bogus. But there’s an even greater cost that you’re paying because what happens if you spend 10 hours memorizing vocabulary? What do you get at the end of the day? At best, you get some vocabulary, right?
What happens if you spend 10 hours reading or listening to comprehensible input? You get vocabulary. You get syntax. You get morphology. If you’re listening, you get phonology. You get semantics. You get knowledge of the culture. You get knowledge of the world. So, I’m getting seven or eight things. I’m getting the whole language system on my side in my 10 hours, and you’re getting vocabulary.
James:
Yeah, I mean, if we talk about patterns because whatever you’re not picking those patterns up, right?
Jeff:
You’re not.
James:
Because that is so essential, isn’t it? Those subconscious choices.
Jeff:
Absolutely. And it’s almost impossible — it is impossible — to learn it explicitly because it’s difficult to understand at an explicit level how a language goes in different patterns. Not having any of that is huge. I remember someone saying, “You can learn all the numbers.” And I immediately thought, “You won’t make the associations.”
James:
Even with something as basic as numbers, no? I don’t know whether you could use that to give a telephone number. I don’t know whether you could use that to say how much you wanted to spend in a shop or whatever.
Jeff:
I don’t think you could because it comes in patterns, right? I think what happens too — and this is just speculation — but I think what happens is that you get a lot of people, language hackers, let’s say, and you’re more familiar with that community than I am. A lot of people will use rote vocabulary instruction or rote vocabulary memorization, or even grammatical memorization. Then they’ll get some comprehensible input. They’ll get a little bit of both. And when they end up being able to communicate and understand, they’ll say, “Look, my memorization and my comprehensible input really helped.” When in fact, the rote memorization didn’t matter at all. It was all about the comprehensible input.
James:
I think it’s a very difficult thing to do because this is what most people online are doing, right? What they’re saying is, “I’ve reflected on my own process.” As if the ability to reflect on something without biases exists, which it doesn’t.
Jeff:
Exactly.
James:
So, that’s another thing, and you’re bringing in all the biases you have. Because you were told at school, either explicitly or implicitly, how a language is learned, so your biases are going to push you toward thinking, “That was the effective thing. That thing matches my schema of what a language class looks like.”
Jeff:
That’s right.
James:
That’s right. A lot of people forget that doing research on yourself is a really hard thing to do.
Jeff:
It’s a good place to start for inspiration, but that’s why we have the scientific method. That’s why we have studies that control for confounds, control for variables, and can tell us whether rote memorization works in reading comprehension — it doesn’t. Whether rote memorization is more efficient than incidental vocabulary acquisition — I would argue it isn’t. That’s why we actually do scientific studies.
Let me say one more point, if I could, about the opportunity cost, the time you wasted spending on vocabulary instruction rather than getting good comprehensible input. There’s hardly any research on this, but I did find one really interesting study by Zhang and Graham. It was a study published in 2019. What they did is they took one class, and they had them listen to a passage that had some unknown words in it. Then they taught them those words explicitly. They did all sorts of vocabulary activities with them, and they did that for over a period of weeks. I think it was six hours.
The other group, the control group, they listened to the passages, and then for the rest of the 45 minutes, or whatever it was — an hour every week — they just listened to the teacher give background information. I’m going to interpret that as they got comprehensible input for six hours. They tested the students on vocabulary and, crucially, on general listening ability before the study and after the study.
Before the study, they were equal, but at the end of the study, the group that had the vocabulary instruction hardly made any progress on their general listening comprehension. The group that got to listen to the teacher talk for six hours made a huge increase, almost one standard deviation increase, on their listening comprehension scores. That’s what I’m talking about when I talk about opportunity costs. Sure, one group learned 10 or 20 more vocabulary words, but the other group got better at phonology, comprehension, syntax — all aspects of the language. And vitally, the other group only got one thing.
James:
Exactly. Because I was speaking to a neuroscientist about people who lose their hearing. The message I got from her was that believability, the ability for your mind to subconsciously fill in gaps, is really important. You don’t listen to every word, but people who lose their hearing can still understand. Why? Because they’ve had so much input that their mind knows what’s coming next. If you haven’t developed that ability to fill in gaps, your listening will never go beyond a certain level.
Jeff:
Yes, exactly. That’s really interesting. I’ll have to go back and watch that interview — I missed that on your channel. Let me just say one more thing, and then I’ll be done in terms of why rote vocabulary memorization is a terrible idea. For most people, it’s boring. I don’t care how committed or dedicated you are to improving your language skills. The average person finds it boring.
When you give, say, college students the opportunity to use rote memorization and say, “Okay, you can use this or not,” ninety percent of them don’t use it. There’s a great study from a couple of years ago on Anki. They took a group of college students and said, “Okay, we set up these vocabulary lists that we want you to study, and it’ll help you with your grade.” I think something like three or four students actually went online and did it consistently. They hated it.
So, the next semester, the researchers said, “Okay, it’s ten percent of your grade. If you don’t go on and use this Anki memorization tool, ten percent will be taken off your grade.” During the first week, like 60-70% of the students went online. The second week, it dropped to 30%, and by the third week, it was down to almost none.
Even when they were being pressured with the possibility of a lower grade, most students didn’t use Anki.
James:
That’s really interesting.
Jeff:
Now, I have to add that the researchers went back and looked at the scores. This was a Spanish class. They found that the students who used Anki did better in the Spanish class. From that, they concluded, “Well, even if most students don’t like using it, if you do buckle down and use it, you’ll get better Spanish scores.”
James:
Is that right?
Jeff:
No, it’s not. This is a classic case of what we call, in research methodology, a “threat to internal validity.” What else could explain your findings? T
he problem with the study is that it wasn’t set up in such a way to differentiate between the students who were eager to get a good grade and, therefore, used Anki, and whether Anki actually helped improve their Spanish scores. The reason those students did better wasn’t because they used Anki; it’s because they were the students who wanted a good grade.
James:
Obviously.
Jeff:
Right, and if you don’t think that’s correct, then you’d have to set up a study where you actually test what you wanted to test, and they didn’t do that. All they really established is that Anki, or whatever you want to call these electronic flashcard programs, are universally unpopular for all but the most… I would say, committed language hackers.
James:
If I can, I just want to ask you one more question that we didn’t plan, but it just occurred to me.
Jeff:
Sure.
James:
All these people who study by memorizing words — this came to my mind because of the motivation thing. I think everyone, especially people who work outside of English-speaking countries, knows students who have phenomenal second language skills because they played video games in English internationally. You know this is a thing. It seems to me you have to go some distance to explain those kids outside of saying this was kind of bred into them. The phenomenon exists. The engagement exists. They’re engaged in these activities, but they’re also doing things like using higher-level thinking skills, analyzing, evaluating — they’re using all these higher-level skills.
Jeff:
Yes, exactly.
James:
And there does seem to be some connection between immersing yourself in this world, using these higher-level thinking skills, and getting these spectacular results.
Jeff:
Yes, I think you’re right. I did a few studies back in the 90s, and Steve Krashen has written about this too — about the overlap between acquisition and what Csikszentmihalyi calls “flow.” The notion of being in flow, of being immersed in something so deeply that you sort of lose track of time. I do think there’s something there. I don’t know exactly what it is, but I think you’re absolutely right. Being immersed in an activity in a second language and being in flow is probably very good for your language acquisition.
I love the notion of people acquiring language from playing video games. I’m too old for that — that came out after my generation — but I think it’s fascinating. Going back to our original point, let’s say about semantic geography, it does seem like because they’re doing more than just understanding, because they’re problem-solving, they’re somehow able to make these connections stronger and faster.
James:
Yes, absolutely. I just wonder how people who talk about memorization might respond to this phenomenon. It seems to me a very difficult one for them to explain.
Jeff:
Yes, I don’t know. There has been some research on video game learning and language acquisition, but I haven’t been too impressed by it. They seem to be caught in a very traditional memorization type of mode. I think it’s really fascinating, though, and I’m not sure how they would respond to it. Probably the response from someone like Paul Nation would be, “Well, I never said that you shouldn’t also immerse yourself.” Typically, someone like Paul will say, “Yes, this is a good idea, and this is a good idea, and this is also a good idea.” Sometimes that’s the best approach because the state of our knowledge is such that we don’t know definitively what the best approach is.
But I think my argument would be that we know enough to know that rote memorization is not a very useful strategy for someone trying to improve their language.
James:
There’s also a not-often-mentioned bias, isn’t there? There’s a bias toward the center. In many walks of life, people say, “Oh, it must be somewhere in the middle.” I did a critical thinking class at the university, and almost every time, they’d come back to me and say, “It’s in the center.” I’d say, “Look, that doesn’t let you off from justifying it.”
Jeff:
Exactly.
James:
But it does seem to be a thing that people do, no?
Jeff:
Well, you know, partly it’s social. You meet these people at conferences. You have lunch with them, dinner with them, you hang out with them in the faculty lounge. People want to get along, to go along, you know? There’s a weird sort of social aspect where you don’t want to stand up at a conference and criticize your colleagues too much. But in fact, that’s what real science is. Real science is going for the jugular in the pursuit of truth, right? That’s what we’re here for. It’s not always in the center.
James:
Okay, well, thank you so much for your contribution to this discussion. It was fascinating speaking to you. I’ll try my best to go through and put as many of those papers as I possibly can. That may be a gradual thing — I may upload and gradually add papers. Thank you so much, Dr. McQuillan. It’s been fascinating.
Jeff:
It’s been my pleasure. Thank you for helping me out again, James. Let’s do it again.
James:
Sure, that’d be great.
Follow Us!