The National Assessment of Educational Progress (NAEP) – “the Nation’s Report Card” – has released results from their latest round of reading tests. These tests are given every two years, but regardless of the results, the reaction in the press is nearly always the same:
“We have a reading crisis in America!“
This panic isn’t based on an actual dramatic drop in scores, either in the past few years or over the past two decades.
In 2019, there was a “statistically significant” decline in fourth grade reading scores from 2017, going from 222 to 220 (on a scale up to 500). This sounds bad, until you remember that when you have huge sample sizes in a study – and NAEP has the largest US sample of schoolchildren – even miniscule changes will be “statistically significant.”
What we really want to know is whether these changes have any practical importance. One way to determine that is to look at the “effect size” change (more on effect sizes here).
Effect sizes can tell us whether changes have any meaningful impact on school achievement. You determine effect sizes by taking the change in scores and dividing it by the standard deviation of those scores.
The change in NAEP fourth grade reading scores from 2017 to 2019 works out to be an effect size of .05.
This is as close to a meaningless change as you can get. The Department of Education classifies any effect size less than .25 as not being educationally significant. Researchers typically class effect sizes of .20 or less as “small,” .50 as “medium,” and .80 or greater as large.
By any standard, then, an effect size of .05 is tiny.
To put it in other terms, a .05 effect size is the equivalent of having a student go from the 50th percentile all the way down to . . . the 48th percentile! A whole two percentile points.
Those of us who’ve been at this for a few years remember the great “reading crisis” of the early 1990s. Back then, we were told that we had a disaster on our hands due to falling NAEP scores.
In 1994, the average fourth grade reading score on the NAEP was 214. This year it’s 220. Scores are now higher than the last reading crisis, yet somehow we are still in crisis.
Everyone just needs to calm down.
But the media and certain players in the education world love this stuff, since they can then find the “cause” of this great decline. In reading, it’s often because we’re not doing enough phonics instruction, as if that would solve anything.
P.S. For the data from the previous literacy “crisis” of the 1990s, see my free, appropriately titled book, The Literacy Crisis, chapter 1.
Follow Us!