{"id":1846,"date":"2019-09-25T13:31:33","date_gmt":"2019-09-25T21:31:33","guid":{"rendered":"http:\/\/backseatlinguist.com\/blog\/?p=1846"},"modified":"2019-09-25T13:31:33","modified_gmt":"2019-09-25T21:31:33","slug":"reading-tests-that-dont-measure-reading-corrected","status":"publish","type":"post","link":"http:\/\/backseatlinguist.com\/blog\/reading-tests-that-dont-measure-reading-corrected\/","title":{"rendered":"Reading Tests That Don’t Measure Reading [CORRECTED]"},"content":{"rendered":"
\n

To make them easier to find, several items originally included in the now-defunct ‘This Week in Language Teaching’ series will be reposted over the next few weeks as separate entries.<\/em><\/p>\n<\/blockquote>\n

Hua and Keenan<\/a> (2017, paywall), following up on early work by Keenan and her colleagues<\/a>, examined five popular reading comprehension tests to see how much of the variance in scores could be explained by a reader’s listening comprehension, and how much by his or her ability to read individual words in isolation (word recognition).<\/p>\n

Hua and Keenan tested a large group of students (N<\/em> = 834) ages 8 to 18. They used everyone’s current favorite for statistical analysis, quantile regression<\/a>, to see how readers of different abilities fared. I report here\u00a0the results for the “average” student:<\/p>\n

    \n
  1. Woodcock-Johnson Passage Comprehension (WJPC):<\/em>
    \nWord Recognition: 56%
    \nListening Comprehension: 33%<\/li>\n
  2. PIAT:<\/em>
    \nWord Recognition: 61%
    \nListening Comprehension: 25%<\/li>\n
  3. Gray Oral Reading Test:<\/em>
    \nWord Recognition: 23%
    \nListening Comprehension: 41%<\/li>\n
  4. Qualitative Reading Inventory-Questions:<\/em>
    \nWord Recognition: 17%
    \nListening Comprehension: 52%<\/li>\n
  5. Qualitative Reading Inventory-Retells:<\/em>
    \nWord Recognition: 17%
    \nListening Comprehension: 52%<\/li>\n<\/ol>\n

    There’s a clear difference between tests #1 and #2 and #3-5: the PIAT and Woodcock-Johnson Passage Comprehension (WJPC) are mostly word recognition tests rather than measures of comprehension.<\/p>\n

    Why the difference? In the PIAT and WJPC, readers read single sentences and\/or complete a “cloze” (fill in the missing word). There is little context to aid the reader, nor is there a very high level of complex comprehension\u00a0required. Tests #3, 4, and 5, however, involve\u00a0reading longer passages and answering questions or retelling the events of the story.<\/p>\n

    In other words, Tests 3-5 measure more of what most people (teachers, the general public) consider “reading comprehension” – understanding what you read.\u00a0Word recognition, on the other hand,\u00a0is often used as a proxy\u00a0for decoding skills (the ability to convert letters into sounds), and not paradoxically is also much more easily influenced by phonics instruction.<\/p>\n

    Why is this important for us to know? At least three reasons:<\/p>\n