The Impact of Automated Writing Evaluation on English as a Foreign Language Learners' Writing Self-efficacy, Self-regulation, Anxiety, and Performance

Aniqa Shah, University of Arizona, Tucson, Arizona, USA

Sari, E., & Han, T. (2024). The impact of automated writing evaluation on English as a foreign language learners’ writing self-efficacy, self-regulation, anxiety, and performance. Journal of Computer Assisted Learning, 40(5), 2065–2080. https://doi.org/10.1111/jcal.13004

Summary of the Article

The reviewed article explored the impact of integrating teacher feedback with automated writing evaluation (AWE) on several key aspects of students' writing development, like self-efficacy, self-regulation, anxiety, and writing performance. The authors described self-efficacy in writing as an individual’s beliefs about their capability to complete a task, while self-regulation as the ability to regulate and manage tasks specific to a writing goal. They defined anxiety with respect to writing as a general hesitation to write in the target language.

The study was conducted with 79 students enrolled in two English as a Foreign Language (EFL) classes at a Turkish university. In a quasi-experimental research design, one class was given feedback from both the teacher and Criterion, a widely recognized AWE tool developed by Educational Testing Service (ETS) that focuses on grammar, usage, and mechanics, while the other class received feedback exclusively from the teacher.

The research utilized a mixed-method approach to gather data. Data was collected in two phases—pre-tests and post-tests—using a suite of five distinct instruments: the Writing Self-Efficacy Scale, a questionnaire on English Writing Self-Regulated Learning Strategies, the Second Language Writing Anxiety Scale, writing tests, and focus group interviews.

Results indicated that students who received combined teacher and automated feedback showed notable improvements in their self-efficacy and writing performance. However, there was no statistically significant difference in the levels of anxiety experienced by students between the two feedback methods. Furthermore, neither feedback approach was found to significantly enhance the use of self-regulated writing strategies compared to the other. Despite these findings, students demonstrated a clear preference for receiving the combined feedback, valuing the automated tool for its ability to identify common grammatical and punctuation errors that they might otherwise overlook. Students appreciated the added confidence this feedback provided, although they also reported some drawbacks, such as the redundancy of the automated feedback and its failure to provide corrective suggestions.

Implications for Writing Instruction

The findings of this study have significant implications for writing instruction, highlighting the potential benefits of incorporating AWE tools into the feedback process. As technology continues to advance, educators have an array of innovative tools at their disposal that can enhance the learning experience and alleviate the burden of providing detailed feedback. The study suggests that AWE tools can serve as an effective supplementary mechanism to teacher feedback, offering preliminary insights into common errors and helping to improve students' self-efficacy and writing performance.

One of the primary objectives of writing instruction is to help students develop self-efficacy in their writing skills (Sun et al., 2021). This study indicates that AWE tools can effectively support this objective. I believe AWE can provide valuable feedback across various writing tasks in all educational levels. For beginner language learners, AWE can assist with basic grammar practice as they construct simple sentences. For advanced learners, who engage in more complex writing processes such as pre-writing, drafting, and proofreading, AWE tools can complement teacher feedback throughout each stage.

The study underscores the importance of addressing the needs of multilingual writers, who often encounter greater challenges in academic writing compared to their monolingual peers (Morton et al., 2015). The integration of AWE tools can offer these students essential support, aiding in the development of their confidence and reduction of anxiety related to their writing skills. In educational contexts where the student population is diverse and English is not the first language for all learners, AWE tools can play a crucial role in assisting those who experience significant self-doubt and fear of judgment. Although AWE tools have limitations, such as providing only limited error descriptions and lacking corrective suggestions, they remain a valuable resource for repeated practice and improvement.

This study also highlights the potential for AWE tools to help manage teachers' workloads. By providing initial feedback on common writing issues, AWE tools can reduce the pressure on teachers to deliver extensive feedback quickly, allowing them to focus on more complex aspects of student writing and overall development. This dual approach can lead to a more efficient and effective feedback process, benefiting both students and educators.

Personal Thoughts

As an English teacher in a second or foreign language context, I consider the integration of AWE tools into writing instruction particularly promising. My decade-long experience teaching English in Pakistan has made me acutely aware of the challenges faced by English language learners, including pervasive self-doubt and anxiety about their writing abilities. In such educational settings, where English is often viewed as the language of prestige (Singh, 2017), AWE tools can provide crucial support by offering immediate, specific feedback in a non-judgmental manner (Tsao et al., 2017).

As an international student in the US, I understand the challenges of adjusting to a new environment, including the demands of mastering academic writing in English. In higher education worldwide, the application of AWE tools can be particularly advantageous. Many educational institutions, especially in Western countries, have a significant population of international students who continue to develop their writing skills in English. For these students, managing academic responsibilities while also improving their English writing skills can be a substantial source of stress. AWE tools can alleviate some of this pressure by providing targeted support to enhance their writing skills. By offering detailed feedback on common errors and helping students refine their language use, these tools can make the adjustment period less overwhelming and support their academic success.

While AWE tools may not completely replace the nuanced feedback provided by teachers, they offer valuable supplementary support that can help students refine their writing skills. The ability to receive detailed feedback on common grammatical and punctuation errors can significantly boost students' confidence and encourage them to engage more actively in the writing process. Moreover, the use of AWE tools can help teachers manage their workload more effectively, allowing them to dedicate more time to addressing complex writing issues and fostering students' overall development.

In conclusion, the study reinforces the potential benefits of combining automated and teacher feedback to enhance writing instruction. By leveraging the strengths of both feedback methods, educators can provide a more comprehensive and supportive learning experience, ultimately contributing to improved writing outcomes and greater student confidence. The integration of AWE tools represents a promising advancement in the field of language education, with the potential to benefit both students and teachers in diverse educational contexts.

References

Morton, J., Storch, N., & Thompson, C. (2015). What our students tell us: Perceptions of three multilingual students on their academic writing in first year. Journal of Second Language Writing, 30, 1–13. https://doi.org/10.1016/j.jslw.2015.06.007

Singh, S. (2017). English as a tool of power, prestige, and growth. International Research Journal of Commerce, Arts and Science, 7(2), 13-18.

Sun, T., Wang, C., Lambert, R. G., & Liu, L. (2021). Relationship between second language English writing self-efficacy and achievement: A meta-regression analysis. Journal of Second Language Writing, 53, 100817. https://doi.org/10.1016/j.jslw.2021.100817

Tsao, J.-J., Tseng, W.-T., & Wang, C. (2017). The effects of writing anxiety and motivation on EFL college students’ self-evaluative judgments of corrective feedback. Psychological Reports, 120(2), 219–241. https://doi.org/10.1177/0033294116687123.

Aniqa Shah is a PhD candidate at the University of Arizona, specializing in literacy with a minor in Second Language Acquisition and Teaching (SLAT). Her research explores language ideologies, linguistic hegemony, issues of language access and equity, the integration of technology in language classrooms, and the intersection of cognition and language.