{"id":39,"date":"2012-01-20T00:00:13","date_gmt":"2012-01-20T08:00:13","guid":{"rendered":"http:\/\/backseatlinguist.com\/blog\/?p=39"},"modified":"2019-11-20T16:48:23","modified_gmt":"2019-11-21T00:48:23","slug":"all-correction-all-the-time-is-written-error-correction-worth-the-effort","status":"publish","type":"post","link":"http:\/\/backseatlinguist.com\/blog\/all-correction-all-the-time-is-written-error-correction-worth-the-effort\/","title":{"rendered":"All Correction, All the Time: Is Written Error Correction Worth the Effort?"},"content":{"rendered":"

Studies reviewed:<\/strong><\/p>\n

\n

Evans, N., K.J. Hartshorn, R. McCollum, & M. Wolfersberger. (2010). Contextualizing corrective feedback in second language writing pedagogy. Language Teaching Research, 14<\/em>(4), 445 -463.<\/p>\n

Hartshorn, K.J., N. Evans, P. Merrill, R. Sudweeks, D Strong-Krause, & N. Anderson. (2010). Effects of dynamic corrective feedback on ESL writing accuracy.\u00a0TESOL Quarterly, 44<\/em>(1), 84-109.<\/p>\n

Evans, N., K.J. Hartshorn, and D. Strong-Krause. (2010). The efficacy of dynamic written corrective feedback for university-matriculated ESL learners accuracy. System, 39, <\/em>229-239.<\/p>\n<\/blockquote>\n

In the world of language teaching, some ideas die hard. \u00a0Error correction is one of those ideas that keeps rising from the dead after each seemingly fatal blow.<\/p>\n

Krashen’s 1984 book, Writing<\/a><\/em>, reviewed the evidence on error correction and found that it rarely did what it promised to do – improve students’ grammatical accuracy. \u00a0Subsequent papers by Krashen and a series of excellent critiques by John Truscott<\/a>\u00a0 (1996, 1999, 2004) put additional nails in the coffin.<\/p>\n

But error correction lives on, with plenty of defenders in the language education field.<\/p>\n

I look here at three studies from researchers at Bingham Young University that take error correction to its logical conclusion – correcting nearly every mistake<\/strong> students make in every essay throughout an entire semester. \u00a0None of the studies, in my view, show that error correction was worth the massive effort that was invested in it. \u00a0I start with descriptions of the studies, then comment on them below.<\/p>\n

Evans, Hartshorn, McCollum, & Wolfersberger (2010)<\/strong><\/p>\n

Participants<\/span>: Two groups ESL students described as “advanced low” studying in an intensive English program (IEP) for a 13-week semester. \u00a0(This was apparently the pilot study for the next two studies, and contained no control group.) \u00a0The groups (N = 12 and 15, respectively) were from different academic semesters, Winter 2007 and Summer 2007, and students ranged in age from 18 to 33.<\/p>\n

Treatment<\/span>: There were six steps in the rather lengthy error correction treatment, which the authors refer to as “dynamic Written Corrective Feedback” (WCF):<\/p>\n

    \n
  1. Each group wrote 10-minute, timed essays at the beginning of most class sessions (the class met Monday through Thursday). \u00a0They were told to “follow the conventions of good paragraph writing, be as linguistically accurate as possible, and make the content substantive” (p. 455). Topics were assigned for each essay. \u00a0On average, the groups in this study wrote 31 paragraphs throughout the semester.<\/li>\n
  2. The teacher collected the essays and then corrected them, returning them the next class session to the students. \u00a0Written feedback consisted of “marking the papers for lexical and syntactic accuracy” using 20 error-correction symbols. \u00a0Citing work by Ferris, the researchers state that the teacher indicated but did not correct “errors students can treat – those that can be corrected with systematic grammar rules” with one of the 20 symbols. But the teacher also did “direct error correction,” actually correcting the errors that students were thought unable to treat, described as “those that result from aspects of the language that must be acquired over time, such as prepositions or some lexical features” (p. 455). Then, the teacher assigned a “holistic” grade, with 75% for accuracy and 25% for content.<\/li>\n
  3. The coded\/corrected paper was returned to the student during the next session. \u00a0The student, in turn, had several additional tasks:
    \n(a) Keep a tally of errors by type of error;
    \n(b) Keep a list of all errors committed in context by category, which consisted of errors “typed exactly as they were originally and erroneously written” (p. 455);
    \n(c) Highlight or underline each error on their typed list of errors; and finally
    \n(d) Edit, type, and resubmit the paragraph to the teacher for a second review.<\/li>\n
  4. The teacher then marked the typed paragraphs again for accuracy, but this time indicating the place where the error occurred with a check mark, a circle, or by underlining, although the teacher could, if needed, also supply the error code again. The papers were then returned to the students.<\/li>\n
  5. Students corrected (if necessary) their second draft and returned it to the teacher.<\/li>\n
  6. The teacher corrected and indicated the errors again, if necessary, and the process repeated until the paragraph was error free. \u00a0The researchers note that most paragraph were error free within two drafts.<\/li>\n<\/ol>\n
    Measure<\/span>: \u00a0Essays were grouped chronologically into four sets for analysis. \u00a0The holistic scores given by the teacher were added and averaged for each set. In addition, writing accuracy was measured by calculating the ratio of error-free clauses to total number of clauses written. The first and last sets’ scores were compared to measure increases in accuracy.<\/div>\n

     <\/p>\n

    <\/div>\n
    <\/div>\n
    Results<\/span>: \u00a0Table 1 shows the results for both the holistic and the error-free clauses measures, showing just the first and last sets of both groups (from Evans et al. Tables 1 and 2, pp. 457-458).<\/div>\n

     <\/p>\n

    Table 1<\/span>: Holistic Scores and Percentage of Error-Free Clauses in Evans et al. (2010)\n\n\n\t\n\t\t\n\t\n\t\t\n\t\t\n\t\t
    <\/th>Holistic<\/th>Error-Free Clauses<\/th>\n\t<\/tr>\n<\/thead>\n
    Group<\/td>1st Set of Essays<\/td>4th Set of Essays<\/td>1st Set of Essays<\/td>4th Set of Essays<\/td>\n\t<\/tr>\n\t
    Winter 2007<\/td>7.39<\/td>7.86<\/td>45%<\/td>55%<\/td>\n\t<\/tr>\n\t
    Summer 2007<\/td>7.45<\/td>7.69<\/td>42%<\/td>54%<\/td>\n\t<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n
    Students improved on their holistic scores from the first set to the fourth set \u00a0for both the Winter 2007 (t = -6.79, df = 11, p<000) and Summer 2007 (t = -4.9, df = 9, p=.001). \u00a0Effect size was measured by partial eta square, which were .81 and .73, respectively. \u00a0According to Cohen (1988), these are large effects. \u00a0Cohen’s d, another, more common measure of effect size, was not calculated, and standard deviations were not provided. \u00a0Wolf (1986) provides us with a formula for calculating d from the t statistic, which here yielded substantial d’s of 4.1 and 3.3 for the increase in the holistic (though largely based on accuracy) scores.<\/div>\n

     <\/p>\n

    <\/div>\n
    <\/div>\n
    Subjects also made apparently large gains on the ratio of error-free clauses to total clauses (Winter 2007 group: t = -3.42, df = 11, p = .006; Summer 2007 group: t = -3.90, df = 9, p = .004). Effect sizes reported by the researchers were partial eta squared (.52 and . 63 for the Winter and Summer groups, respectively). \u00a0Cohen’s d<\/em> calculated from the t statistic were also large, with d<\/em> = \u00a02.06 for the Winter group and 2.6 for the Summer group.<\/div>\n

     <\/p>\n

    <\/div>\n
    <\/div>\n

    Hartshorn et al. (2010):\u00a0<\/strong><\/p>\n

    Participants<\/span>: 47\u00a0 low- and mid-advanced university Intensive English Program (IEP) students (mean age: 24 years) of various language backgrounds.<\/p>\n

    Treatment<\/span>:\u00a0 Students in the treatment group underwent substantially the same process of total error correction as in the previously discussed study of Evans et al.: 10-minute essays in each class, errors coded by the teacher, tallied and corrected by the student, and so on until the essay was free of errors. \u00a0In addition, the teacher discussed common errors in class.<\/p>\n

    The control group participated in a \u201cprocess writing\u201d approach during the 15 week period, during which, however, errors were also corrected. They did not do short, timed compositions like the experimental group, but wrote four multi-draft term papers and received feedback on each draft.\u00a0 The total amount of writing, according to the researchers, was roughly the same.<\/p>\n

    Measure<\/span>: The pretest and posttest consisted of 30-minute typed essays rather than the in-class essays. \u00a0Writing accuracy was measured by the ratio of error-free\u00a0T-units<\/a>\u00a0to total T-units.\u00a0Also measured were \u201crhetorical competence\u201d with a rubric adapted from the TOEFL iBT; writing fluency, defined as the number of total words; and writing complexity, defined as the mean number of words per T-unit.<\/p>\n

    Results<\/span>: The treatment and control groups did not differ significantly on the rhetorical competence, fluency, or complexity measures, although the contrast group wrote slightly more than the control group (Cohen\u2019s d<\/em> is not reported; partial eta squared was .07). The treatment group outscored the contrast group on accuracy, however, by what appears to be a wide margin.\u00a0 The researchers somewhat confusingly report the results as whole numbers, although they are in fact ratios of error-free T-units to total T-units. Results are summarized in Table 2, but using percent of error-free T-units for the pre- and posttest calculated from the ratios given in study’s Table 4 (p. 99).<\/p>\n

    Table 2<\/span>: Percent of Error-Free T-units for Experimental and Contrast Groups in Hartshorn et al.\u00a0\n\n\n\t\n\t\t\n\t\n\t\t\n\t\t
    <\/th>Pretest<\/th>Posttest<\/th>Difference<\/th>\n\t<\/tr>\n<\/thead>\n
    Experimental<\/td>14.02%
    \r\n(15.0)<\/td>    		24.16%
    \r\n(19.46)<\/td>    		10.14<\/td>\n\t<\/tr>\n\t
    Control<\/td>16.3%
    \r\n(10.7)<\/td>    		13.78%
    \r\n(11.81)<\/td>    		-2.52<\/td>\n\t<\/tr>\n<\/tbody>\n<\/table>\n<\/p>\n

    Effect size: partial eta squared = .21; Cohen\u2019s d <\/a>= .64 (calculated from Table 4, p. 99, using pooled standard deviation)<\/p>\n

    The researchers conclude that WCF was effective in significantly improving writing accuracy, noting that the effect size as measured by partial eta squared was large (using Cohen\u2019s (1988) guidelines). Cohen’s d\u00a0<\/em>was not calculated in the original article. It was .64, which is considered by Cohen a medium effect. In terms of the practical significance of the results, \u00a0Hartshorn et al. note that the treatment group\u2019s writing was \u201cjust over 75% more accurate than the writing of the contrast group.\u201d<\/p>\n

    Evans, Hartshorn, & Strong-Krause (2010):<\/strong><\/p>\n

    Participants<\/span>: Both the treatment group (N = 14) and the contrast group (N = 16) were university ESL students (mean age: 21).<\/p>\n

    Treatment<\/span>: Both treatment and control groups received essentially the same semester-long curriculum as the treatment and control groups of Hartshnorn et al. (2010) and Evans et al. (2010). \u00a0The treatment group received extensive error correction, including having to code, tally, and log all of their errors, as well as rewrite their 10-minute in-class essays over until there were error free, a process that the researchers note \u201cmany times\u2026required multiple drafts\u201d (p. 234). \u00a0The 10-minute essays were written \u201cthree to four times per week\u201d over the 13-week semester, totally about 19 pages of \u201cpolished\u201d writing.\u00a0 The contrast group received a process-writing approach, although their errors were also corrected, but not as consistently or extensively as the treatment group.<\/p>\n

    Measure<\/span>: Fluency and complexity were measured in a similar manner as in Hartshorn et al. (2010) (number of total words written and mean number of words per clause).\u00a0 As in Evans, Hartshorn, McCollum, and Wolfersberger, the accuracy measure was changed from the ratio of error-free clauses over total clauses, since it was thought to provide a more sensitive measure of change than T-unit analysis.<\/p>\n

    Results<\/span>: Both fluency and complexity of the treatment group was slighly worse than the contrast group, although the researchers report that the effect sizes were relatively small (partial eta squared .04 and .06, respectively; Cohen’s d was not reported). On the measure of accuracy, the treatment group outscored the comparison students.\u00a0 Results are reported in Table 3, from Evans et al.\u2019s Table 3 (p. 235).<\/p>\n

    Table 3<\/span>: Percent of Error-Free Clauses for Experimental and Contrast Groups in Evans et al. \n\n\n\t\n\t\t\n\t\n\t\t\n\t\t
    <\/th>Pretest<\/th>Posttest<\/th>Difference<\/th>\n\t<\/tr>\n<\/thead>\n
    Experimental<\/td>47.10%
    \r\n(11.2)<\/td>    		57.80%
    \r\n(10.9)<\/td>    		10.7<\/td>\n\t<\/tr>\n\t
    Control<\/td>51.40%
    \r\n(12.6)<\/td>    		50.30%
    \r\n(19.4)<\/td>    		-1.1<\/td>\n\t<\/tr>\n<\/tbody>\n<\/table>\n<\/p>\n

    Effect size: partial eta squared = .16; Cohen\u2019s d = .48 (calculated from Table 3, p. 235)<\/p>\n

    Comments<\/span>:<\/p>\n

    1. As with many studies on error correction and “focus on form,” the conditions for Monitor use appear to be met during the elicitation measures of all three studies. The Monitor (Krashen, 1982) is our ability to use conscious knowledge (learning) of the language (such as grammar rules) to make our speaking and writing more accurate than what it might otherwise be.<\/p>\n

    To use the Monitor, you must:<\/p>\n

    (1) know the rule you need to apply in the given situation,
    \n(2) be focused on the form or accuracy of the sentence you are producing, and
    \n(3) have time to use the rule.<\/p>\n

    These conditions are difficult to meet in the real world, but, when the conditions are<\/em> met, the Monitor can improve our accuracy. In these three studies, as we would expect, the students who knew the rule, were focused on form and accuracy, and had sufficient time to use their conscious knowledge (learning) were more accurate than (in the two studies with control groups) students less focussed on form and presumably less knowledgeable of the rules.<\/p>\n

    The results of these studies, then, are perfectly consistent with current second language acquisition theory (Krashen, 2003) on the use of conscious knowledge in language production.\u00a0 There is little doubt that after 13-15 weeks of massive error correction and perhaps as many as 30 hours spent focussing on form, not only in class but in their homework, students already oriented to grammar instruction and error correction (university ESL\/IEP students) were able to use their conscious knowledge during the assessments.<\/p>\n

    While the essays were timed in all three studies, the 10-minute essays in the first study were very short pieces of writing in which accuracy was heavily stressed (the sample student essay included in the article is only eight sentences and less than 100 words long, although no measures of fluency were reported to know how typical this sample was). For the other two studies, 30 minutes would be for most students sufficient time to focus on the grammatical correctness of their writing.<\/p>\n

    In addition to meeting the conditions of the Monitor, the treatment groups were also very practiced in timed essay writing, having done so 30 or more\u00a0times during the semester (!).\u00a0 There is no indication that the contrast groups did any timed writings, and were thus likely to be less practiced at writing under such conditions.<\/p>\n

    2. As the researchers point out, this was not a comparison of error correction and no error correction, but (in studies 2 and 3 above) extreme corrective feedback with more traditional error correction. Other studies, such as Syying Lee’<\/a>s work on extensive reading and writing (Lee & Hsu, 2009) have found accuracy improves significantly simply through more reading, without the extensive (and time-consuming) WCF used here.<\/p>\n

    3. The studies lacked any delayed post-test.\u00a0 This is a crucial element in any study on the effects of form-focused instruction, as previous studies which have included a delayed post-test have nearly always found a sharp decline in the gains demonstrated on the immediate post-test.<\/p>\n

    The effect of explicit instruction typically fades, although Krashen (2003) notes that it can take up to several months for the declines to show up (p. 42-43). This weakness of all three studies is alone reason to question the researchers’ optimistic assessment of the benefits of corrective feedback.<\/strong><\/p>\n

    The gains that were found were indeed large by Cohen\u2019s guidelines for partial eta squared, but, for the last two studies, somewhat less so when calculated from the mean post-test scores (d = .64, a medium to large effect for Hartshorn et al., d = .48, a small to medium effect for third study).<\/p>\n

    4. Unlike the researchers, I do not find the practical effect<\/strong> of this huge investment of time in error correction very impressive. \u00a0In the first study, students went from having 55-58% of their clauses with errors to 45-46% with errors. In Hartshorn et al., treatment group students went from having 86% of their T-units with errors to \u201conly\u201d 76% error-filled T-units. \u00a0In the third study, students in the treatment group went from having 53% of their clauses with errors to 42% with errors, all this after painstaking and massive error correction efforts.<\/p>\n

    While this is an improvement, it is dubious whether teachers would think the considerable effort involved in carrying out this \u201call-correction, all-the-time\u201d agenda in their own classrooms worth it for such a small result.<\/p>\n

    Remember that the teachers coded nearly every error in the essays from nearly every class period, in addition to having to re-code\/correct follow-up essays. Students had to re-type the errors, tally them, classify them, underline the error, type and correct the essay, then repeat the process if the essay wasn’t error free.<\/p>\n

    No estimates are given of how much time the teachers and students collectively spent on correcting errors, but is appears to be substantial. These of course are, as pointed out, students coming from a population already heavily geared toward grammar study, making them ideal candidates for this sort of treatment. To leave the semester still making so many errors can hardly be claimed as a victory for error correction.<\/p>\n

    Works Cited<\/strong><\/p>\n

    Cohen, J. (1988). Statistical power analysis for the behavioral sciences<\/em> (2nd ed). Hillsdale, NJ: Lawrence Erlbaum.<\/p>\n

    Krashen, S.D. (1982). Principles and practice in second language acquisition<\/em>. Pergamon Press.<\/p>\n

    Krashen, S.D. (1984). Writing: Research, theory, and applications<\/em>. Torrance, CA: Laredo Publishing.<\/p>\n

    Krashen, S.D. (2003). Explorations in language acquisition and use: The Taipei Lectures<\/em>. Portsmouth, NH: Heinemann.<\/p>\n

    Lee, S. Y., & Hsu, Y. Y. (2009).\u00a0Determining the crucial characteristics of Extensive Reading programs: The Impact of Extensive Reading on EFL writing<\/a>.\u00a0The International Journal of Foreign Language Teaching (IJFLT)\u00a0<\/em>, 12-20.<\/p>\n

    Truscott, J.\u00a0(1996).\u00a0The case against grammar correction in L2 writing classes.<\/a>\u00a0Language Learning, 46,\u00a0<\/em>327-369.<\/p>\n

    Truscott, J. (1999).\u00a0The case for “The case against grammar correction in L2 writing classes” A response to Ferris.<\/a>\u00a0Journal of Second Language Writing, 8,\u00a0<\/em>111-122.<\/p>\n

    Truscott, J.\u00a0(2004).\u00a0Evidence and conjecture on the effects of error correction: A response to Chandler.<\/a>\u00a0Journal of Second Language Writing, 13,\u00a0<\/em>337-343\u00a0.<\/em><\/p>\n

    Wolf, F. (1986). Meta-analysis: Quantitative methods for research synthesis<\/em>. Newbury Park, CA: Sage Publications.<\/p>\n

     <\/p>\n

     <\/p>\n

     <\/p>\n

     <\/p>\n

     <\/p>\n

     <\/p>\n

     <\/p>\n\n","protected":false},"excerpt":{"rendered":"

    Studies reviewed: Evans, N., K.J. Hartshorn, R. McCollum, & M. Wolfersberger. (2010). Contextualizing corrective feedback in second language writing pedagogy. Language Teaching Research, 14(4), 445 -463. Hartshorn, K.J., N. Evans, P. Merrill, R. Sudweeks, D Strong-Krause, & N. Anderson. (2010). Effects of dynamic corrective feedback on ESL writing accuracy.\u00a0TESOL Quarterly, 44(1), 84-109. Evans, N., K.J. […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[6],"tags":[],"_links":{"self":[{"href":"http:\/\/backseatlinguist.com\/blog\/wp-json\/wp\/v2\/posts\/39"}],"collection":[{"href":"http:\/\/backseatlinguist.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/backseatlinguist.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/backseatlinguist.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/backseatlinguist.com\/blog\/wp-json\/wp\/v2\/comments?post=39"}],"version-history":[{"count":64,"href":"http:\/\/backseatlinguist.com\/blog\/wp-json\/wp\/v2\/posts\/39\/revisions"}],"predecessor-version":[{"id":2057,"href":"http:\/\/backseatlinguist.com\/blog\/wp-json\/wp\/v2\/posts\/39\/revisions\/2057"}],"wp:attachment":[{"href":"http:\/\/backseatlinguist.com\/blog\/wp-json\/wp\/v2\/media?parent=39"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/backseatlinguist.com\/blog\/wp-json\/wp\/v2\/categories?post=39"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/backseatlinguist.com\/blog\/wp-json\/wp\/v2\/tags?post=39"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}