Email this article 
Artificial intelligence in education: Effects of using integrative automated writing evaluation programs on honing academic writing instruction
Abstract
Automated writing evaluation (AWE), which is the result of educational artificial intelligence technology, is a process of scoring and evaluating learners’ written texts automatically. The current study examined the effects of using integrative AWE programs on honing academic writing instruction. It also assessed students’ perceptions towards using these programs. A quasi-experimental pretest-posttest two-group design was used. Test, questionnaire, focus group discussion, and teacher diary were used to collect data from 92 randomly selected participants. The experimental group students learned writing skills with Writerly and Google Docs in integration, but the control group students learned through the conventional paper and pencil feedback system. When the quantitative data were analyzed through independent samples T-test and descriptive statistics, the qualitative data were analyzed thematically. The findings revealed that using the integrated AWE programs honed academic writing instruction because there was a statistical difference between the experimental and control groups in their academic writing performance. Hence, students who learned using the integrated AWE programs honed their academic writing performance because they were able to produce essays that addressed task achievement, coherence and cohesion, lexical resource, grammatical range and accuracy. However, students who learned through the conventional method were less effective in producing quality essays. Besides, the findings also discovered that the experimental group students had positive perceptions towards using the aforementioned AWE programs because they found the programs interesting, effective, goal-oriented, and supportive. Consequently, this study recommends researchers, curriculum designers, instructional material designers, teachers, and students pay due attention to integrated AWE programs.
Keywords
Full Text:
PDFReferences
Aken, A. (2017). An Evaluation of Assessment-Oriented Computer-Based Text Analysis Paradigms. Higher Education Research, 2(4), 111–116. https://doi.-org/10.11648/j.her.20170204.12
Ariyanto, M.S., Mukminatien, N., and Tresnadewi, S. (2021). College students’ perceptions of an automated writing evaluation as a supplementary feedback tool in a writing class. Jurnal Ilmu Pendidikan (JIP), 27(1), 41-51.
Asratie, M.G., Wale, B.D. and Aylet, Y.T. (2022). Effects of using educational technology tools to enhance EFL students’ speaking performance. Education and Information Technologies. https://doi.org/10.1007/s10639-022-11562-y
Attali, Y., & Burstein, J. (2006). Automated essay scoring with e-rater v.2. Journal of Technology, Learning, and Assessment, 4(3),1-30.
Birhan, A. T. (2018). Effects of Mastery Learning Instruction on Engineering Students’ Writing Skills Development and Motivation. Journal of Language and Education, 4(4), 20–30. https://doi.org/10.17323/2411-7390-2018-4-4-20-30.
British Council. (2018). IELTS Task 2 Writing Band Descriptors: Public Version.
Chen, C.E. and Cheng, W.E. (2018). Beyond the Design of Automated Writing Evaluation: Pedagogical Practices and Perceived Learning Effectiveness in EFL Writing Classes. Language Learning & Technology, 12 (2), 94-112.
Chen, Z., Chen, W., Jia, J., & Le, H. (2022). Exploring AWE-supported writing process: An activity theory perspective. Language Learning & Technology, 26(2), 129–148. https://doi.org/10125/73482
Cheng, W. Y. (2006). The Use of a Web-based Writing Program in College English Writing Classes in Taiwan: A Case Study of MyAccess. Unpublished Master’s thesis. National Kaohsiung First University of Science and Technology, Taiwan.
Cotos, E. (2014). Genre-Based Automated Writing Evaluation for L2 Research Writing: From Design to Evaluation and Enhancement. Palgrave Macmillan. United States.
Creswell, J.H. (1994). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. (4th Ed.). SAGE Publications, Inc.
Derseh, B. (2020). Enhancing EFL students’ writing performance through inquiry-based learning. Italian Journal of Educational Research, XIII:40, 138-156. DOI 10.7346/SIRD-012020-P138.
El Ebyary, K. & Windeatt, S. (2010) The impact of computer-based feedback on students’ written work. International Journal of English Studies, 10(2): 121–142. https://files.eric.ed.gov/fulltext/EJ936915.pdf https://doi.org/10.6018/ijes/2010/2/119231
Fan, N. and Ma, Y. 2022). The Effects of Automated Writing Evaluation (AWE) Feedback on Students’ English Writing Quality: A Systematic Literature Review. Language Teaching Research Quarterly, (28), 53-73.
Foltz, P.W., Laham, D., and Landauer, T.K. (1999). The Intelligent Essay Assessor: Applications to Educational Technology. Interactive Multimedia Electronic Journal of Computer-Enhanced Learning, 1, 2, http://imej.wfu.edu/articles/1999/2/04/index.asp.
Jingxin, G. and Razali, A.B. (2020). Tapping the Potential of Pigai Automated Writing Evaluation (AWE) Program to Give Feedback on EFL Writing. Universal Journal of Educational Research, 8 (12B): 8334-8343. DOI: 10.13189/ujer.2020.082638
Lane, H.C., Yacef, K., Mostow, J. and Pavlik, P. (2013). Artificial intelligence in education: 16th International Conference, Memphis, TN, USA, July 9-13 Proceedings.
Lee, I. (2019). Teacher written corrective feedback: Less is more. Language Teaching, 52(4), 524-536. https://doi.org/10.1017/S0261444819000247
Li, Z., Feng, H.-H. & Saricaoglu, A. (2017) The short-term and long-term effects of AWE feedback on ESL students’ development of grammatical accuracy. CALICO Journal, 34(3): 355–375. https://journals.equinoxpub.com/index.php/CALICO/article/view/26382
Liao, H.-C. (2016) Using automated writing evaluation to reduce grammar errors in writing. ELT Journal, 70(3): 308–319. https://doi.org/10.1093/elt/ccv058
LinHuang, S. H. (2010). The exploitation of e-writing in an EFL classroom: Potential and challenges. Unpublished Master’s thesis. I-Shou University, Taiwan.
Page, E. (2003). Project essay grade: PEG. In M. D. Shermis & J. Burstein (Eds.), Automated essay scoring: A cross-disciplinary perspective, 43-54. Mahwah, NJ: Lawrence Erlbaum Associates.
Piaget, J. (1973). To understand is to invent: The future of education (G. Roberts, Trans.). NY: Grossman Publishers.
Ranalli, J., Link, S., & Chukharev-Hudilainen, E. (2016). Automated writing evaluation for formative assessment of second language writing: Investigating the accuracy and usefulness of feedback as part of argument-based validation. Educational Psychology, 37(1), 8-28.
Saricaoglu, A., and Bilki, Z. (2021). Voluntary use of automated writing evaluation by content course students. ReCALL, 33: 3, 265–277. doi:10.1017/S0958344021000021
Seyyedrezaie, Z.S., Ghonsooly, B., Shahriari, H., and Fatemi, A.H. (2016). A Mixed Methods Analysis of the Effect of Google Docs Environment on EFL Learners’ Writing Performance and Causal Attributions for Success and Failure. Turkish Online Journal of Distance Education-TOJDE, 17(3), 7.
Shermis, M.D. and Burstein, J. (2003). Automated essay scoring: A cross-disciplinary perspective,147-167. Mahwah, NJ: Lawrence Erlbaum.
Wale, B.D. and Bishaw, K.S. (2020). Effects of using inquiry-based learning on EFL students’ critical thinking skills. Asian-Pacific Journal of Second and Foreign Language Education, 5:9. https://doi.org/10.1186/s40862-020-00090-2
Wale, B.D. and Bogale, Y.N. (2021). Using inquiry-based writing instruction to develop students’ academic writing skills. Asian-Pacific Journal of Second and Foreign Language Education, 6:4. https://doi.org/10.1186/s40862-020-00108-9
Wang, P. (2013). Can Automated Writing Evaluation Programs Help Students Improve their English Writing? International Journal of Applied Linguistics & English Literature, 2, (1).
Yang, N. D. (2004). Using MyAccess in EFL writing. The proceedings of 2004 International Conference and Workshop on TEFL & Applied Linguistics (pp. 550-564). Taipei, Taiwan: Ming Chuan University.
Zhang, S. (2021). Review of automated writing evaluation systems. J. China Comput. Assist. Lang. Learn, 1(1): 170–176. https://doi.org/10.1515/jccall-2021-2007
Zhanga, L. and Huang, Z. (2020). Effects of an Automated Writing Evaluation System on Students’ EFL Writing Performance. So, H. J. et al. (Eds.). Proceedings of the 28th International Conference on Computers in Education. Asia-Pacific Society for Computers in Education.
DOI: https://doi.org/10.21831/cp.v43i1.67715
Refbacks
- There are currently no refbacks.
Social Media:
Jurnal Cakrawala Pendidikan by Lembaga Pengembangan dan Penjaminan Mutu Pendidikan UNY is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Based on a work at https://journal.uny.ac.id/index.php/cp/index.
