Item analysis of pre-service EFL teachers’ formative test in teaching practicum for program evaluation

Neti Hartati

doi:10.21831/lt.v11i2.67170

Item analysis of pre-service EFL teachers’ formative test in teaching practicum for program evaluation

Neti Hartati, Universitas Muhammadiyah Prof. Dr. Hamka, Indonesia

10.21831/lt.v11i2.67170

Abstract

Teaching practicum is crucial for pre-service EFL teachers' professional development. Many studies have investigated student-teachers' performance in teaching practicum in various aspects. However, research on their performance in constructing test items in teaching practicum has not been found. Test construction is an essential pedagogical competence that student-teachers must master. Investigating their performance in test construction may give valuable feedback to student-teachers and for teacher education. To fill the research gap, this study conducted an item analysis of a formative test constructed by two student-teachers in a teaching practicum. It revealed that the DP of the items have good quality but the FV analysis showed there is no difficult item suggesting that they cannot make difficult questions. The test reliability is high (0.900625) but there are 6 invalid items. Three items contain grammatical errors creating students' confusion influencing the FV, DP, and the validity. It revealed that the test makers' grammar mastery may determine item quality. It suggests student-teachers improve their grammar mastery, and teacher education improves the quality of English grammar teaching and refine their curriculum of evaluation and language testing subjects and include the evaluation of student-teachers' performance in language assessment in teaching practicum program as a focus.

Keywords

item analysis; short-essay questions; formative test; pre-service English teachers; evaluation

References

Alahmadi, N., Alrahaili, M., & Alshraideh, D. (2019). The Impact of the Formative Assessment in Speaking Test on Saudi Students’ Performance. Arab World English Journal, 10(1), 259–270.

Anam, S., & Putri, N. V. (2021). How Literate am I about Assessment: Evidence from Indonesian EFL Pre-service and In-service Teachers. ENGLISH REVIEW: Journal of English Education, Volume 9, Issue 2, June, 377-388.

Arifin, Z. (2012). Evaluasi Pembelajaran. PT. Remaja Rosdakarya.

Arikunto, S. (2018). Dasar-dasar Evaluasi Pendidikan, edisi 3. Bumi Aksara.

Astuti, Y. D., & Drajati, N. A. (2022). Teaching Practicum Experiences: Pre-service English Teachers' Self-Reflections of Their Professional Growth. Journal of Innovation in Educational and Cultural Research, Vol 3 (3), 382-389.

Brookhart, S. M. (2001). The standards and Classroom Assessment research. In Paper presented at the annual meeting of the American Association of Colleges for Teacher Education (p. 15). ERIC Document Reproduction Service No. ED451189.

Brown, H. D. (2003). Language Assessment: Principles and Classroom Practices. Pearson Longman.

CETL. (2024, November 6). Critical Thinking and other Higher-Order Thinking Skills. Retrieved from Center for Excellence in Teaching and Learning: https://cetl.uconn.edu/resources/design-your-course/teaching-and-learning-techniques/critical-thinking-and-other-higher-order-thinking-skills/

Chan, K. T. (2021). Embedding Formative Assessment in Blended Learning Environment: The Case of Secondary Chinese Language Teaching in Singapore. Education Sciences, 11(360), 1-12.

Darmawan, M., Sudarsono, Riyanti, D., Yullana, Y. G., & Sumarni. (2022). A Test Items Analysis of English Teacher Made Test. Journal of English Education and Teaching (JEET), Volume 6, number 4, 498-513.

EDC. (2013, September 22). Understanding Evaluation: promote Prevent-3 Bold Steps. Retrieved from Education Development Centre, Inc: http://positiveschooldiscipline.promoteprevent.org/

Eksi, G. Y., & Yakisik, B. (2016). To Be Anxious or Not: Student Teachers in the Practicum. Universal Journal of Educational Research (46), 1332-1339.

Harmer, J. (2015). The Practice of English Language Teaching 5th edition. Pearson.

Hartati, N., & Yogi, H. P. (2019). Item Analysis for a Better Quality Test. English Language in Focus (ELIF), 59-69.

Hendriwanto. (2021). A Reflective Teaching Practicum as a Platform for Stimulating Teachers' Professional Development. Journal of Education for Teaching, 1-3.

Houston, D., & Thompson, J. N. (2017). Blending Formative and Summative Assessment in a Capstone: 'It's not your tools, it's how you use them'. Journal of University Teaching & Learning Practice, Vol. 14, Issue 3, , 1-13, http://ro.uow.edu.au/jutlp/vol14/iss3/2.

Hussein, A., & Razeq, A. (2022). Teachers Candidates' Experience in a Practicum for English as a Foreign Language. International Journal on Studies in English Language and Literature (IJSELL), Vol. 11, Issue 2, 2347-3134.

Ismail, S., Rahul, D. R., Patra, I., & Rezvani, E. (2022). Formative vs. Summative Assessment: Impacts on Academic Motivation, Attitude toward Learning, Test Anxiety, and Self regulation Skill. Language Testing in Asia, 1-23, https://doi.org/10.1186/s40468-022-00191-4.

Jeong, H. (2013). Defining Assessment Literacy: Is it different for language testers and non-language testers? Language Testing, 30(3), 345-362. https://doi.org/10.1177/0265532213480334.

Karim, S. A., Sudiro, S., & Sakinah, S. (2021). Utilizing Test Items Analysis to Examine the Level of Difficulty and Discriminating Power in a Teacher-made Test. EduLite Journal of English Education, Literature, and Culture, 256-269.

Kim, J. (2020). Learning and Teaching Online During Covid-19: Experiences of Student Teachers in an Early Childhood Education Practicum. International Journal of Early Childhood 52(2), 145-158.

Kissi, P., Baidoo-Au, D., & Anane, E. (2023). Teachers' Test Construction Competencies in Examination-oriented Educational System: Exploring Teachers' Multiple-choice Test Construction Competence. Frontiers in Education, Volume 8, 1-14. https://doi.org/10.3389/feduc.2023.1154592.

Kosar, G. (2021). Distance Teaching Practicum: Its Impact on Pre-service EFL Teachers' Preparedness for Teaching. IAFOR Journal of Education, vol 9 (2), 111-126.

Lam, R. (2019). Teacher Assessment Literacy: Surveying knowledge, conceptions and practices of classroom-based writing assessment in Hong Kong. System, 81(1), 78-89. https://doi.org/10.1016/j.system.2019.01.006.

Lestari, A. E., & Lestari, S. (2022). Pre-service English Teachers' Practices of Designing and Implementing Lesson Plans for Teaching Practicum. LingTera, 9(1), 25-36.

Lin, S. (2018). Item Analysis of English Grammar Achievement Test. Mandalay University of Foreign Languages Research Journal, Vol. 9, No. 1, 13-20.

Liu, F., Vadivel, B., Rezvani, E., & Namaziandost, E. (2021). Using Games to Promote EFL Learners’ Willingness to Communicate (WTC): Potential Effects and Teachers’ Attitude in Focus. Frontiers in Psychology, 12 (762447), 1-10.

Mahmoudi, F., & Ozkan, Y. (2016). Practicum Stress and Coping Strategies of Pre-service English Language Teachers. Procedia - Social and Behavioral Sciences 232: International Conference on Teaching and Learning English as an Additional Language, 494-501.

Masita, M., & Fitri, N. (2020). The Use of Plickers for Formative Assessment of Vocabulary Mastery. Ethical Lingua Journal of Language Teaching and Literature, 7(2), 311–320.

Musial, D., Nieminen, G., Thomas, J., & Burke, K. (2009). Foundations of Meaningful Educational Assessment. McGraw-Hill Higher Education.

Nemati, M., Alavi, S. M., Mohebbi, H., & Masjedlou, A. P. (2017). Teachers' Writing Proficiency and Assessment Ability: The Missing Link in Teachers' Written Corrective Feedback Practice in an Iranian EFL Context. Language Testing in Asia, 7(1). https://doi.org/10.1186/s40468-017-0053-0.

Nicol, D. J., & Macfarlane-Dick, D. (2006). Formative Assessment and Self-regulated Learning: A Model and Seven Principles of Good Feedback Practice. Studies in Higher education, Vo. 31. No. 2, April, 199-218.

Nurjannah, & Lestari, S. (2021). The Teaching Practicum Experience of Pre-service English Language Teachers through Synchronous Learning. EDULINK (Education and Linguistics Knowledge Journal), vol. 3. (2), 92-115.

Ozan, C., & Kıncal, R. Y. (2018). The Effects of Formative Assessment on Academic Achievement, Attitudes toward the Lesson, and Self-regulation Skills. Educational Sciences: Theory and Practice, 18, 85-118.

Pasaka, R. W., Emilia, E., & Amalia, L. L. (2014). The Exploration of Pre-service EFL Teachers' Challenges in Field Practicum. International Journal of Education, Vol. 7, No. 2, May, 144-153.

Popham, W. J. (2001). The Truth about Testing: An educator's call to action. ASCD (Association for Supervision and Curriculum Development).

Qian, D. D. (2014). School-based English Language Assessment as a High-stakes Examination Component in Hong Kong: Insights of Frontline Assessors. Assessment in Education: Principles, Policy & Practice, 21(3), 251-270. https://doi.org/10.1080/0969594X.2014.915207.

Rachmawati, D., Emilia, E., & Lukmana, I. (2017). Self-concept of EFL Pre-service Teachers: A Reflection from a Teacher Practicum in Indonesia Context. The Journal of English Language Studies, Vol. 02, No. 01,, 1-18.

Rahayuningsih, D. (2016). Student Teachers' Challenges in Developing Teaching Materials During Teaching Practicum in Vocational School. Journal of English and Education 4(2), 24-34.

Rezai, A., Namaziandost, E., Miri, M., & Kumar, T. (2022). Demographic Biases and Assessment Fairness in Classroom: Insights from Iranian University Teachers. Language Testing in Asia, 12(1), 1–20. https://doi.org/10.1186/s40468-022-00157-6.

Riyanti, D. (2020). Students' Reflections in Teaching Practicum: A case study of EFL Pre-service Teachers. Journal on English as a Foreign Language, Vol. 10 (20), 268-289.

Saricoban, A. (2010). Problems Encountered by Student-teachers during Their Practicum Studies. Procedia Social and Behavioral Sciences 2, 707-711.

Schafer, W. D. (1993). Assessment in Teacher Education. Theory into Practice, 32(2), 118-126.

Schafer, W. D., & Lissitz, R. W. (1987). Measurement Training for School Personnel: Recommendations and reality. Journal of Teacher Education, 38(3), 57-63.

Stiggins, R. J. (1999). Assessment, Student Confidence, and School Success. Phi Delta Kappan, 81(3), 191-198.

Sudijono, A. (2012). Pengantar Evaluasi Pendidikan. Rajawali Press.

Sumarsono, S. (2020). Metode Penelitian. UHAMKA.

Toksoz, S., & Ertunc, A. (2017). Item Analysis of a Multiple-Choice Exam. Advances in Language and Literacy Studies, Volume 8, Issue 6, 141-146. http://dx.doi.org/10.7575/aiac.alls.v.8n.6p.140.

Tsagari, D., & Vogt, K. (2017). Assessment Literacy of Foreign Language Teachers around Europe: Research, challenges and future prospects. Language Testing and Assessment, 6(1), 41-61.

Umam, A., & Indah, Y. A. (2020). Exploring In-Service TEYL Teachers' Assessment Literacy: Implication for Continuing Professional Development. Journal of English Educators Society, 5(1), 47-51. https://doi.org/10.21070/jees.v5il.364.

Vadivel, B., Namaziandost, E., & Saeedian, A. (2021). Progress in English Language Teaching through Continuous Professional Development—Teachers’ Self-Awareness, Perception, and Feedback. Frontiers in Education, 6, 757285, 1-10, https://doi.org/10.3389/feduc.2021.757285.

Vogt, K., Tsagari, D., & Csépes, I. (2020). Linking Learners’ Perspectives on Language Assessment Practices to Teachers’ Assessment Literacy Enhancement (TALE): Insights from Four European Countries. Language Assessment Quarterly, 1-24, https://doi.org/10.1080/15434303.2020.1776714.

Wallace, M. J. (1991). Training Foreign Language Teachers: A Reflective Approach. Cambridge University Press.

William, D. (2011). Embedded Formative Assessment. Solution Tree Press.

Wise, S. L., Lukin, L., & Roos, L. (1991). Teacher Beliefs about Training in Testing and Measurement. Journal of Teacher Education, 42(1), 37-42.

Yan, X., Zhang, C., & Fan, J. J. (2018). "Assessment knowledge is important, but ...": How contextual and experiential factors mediate assessment practice and training needs of language teachers. System, 74, 158-168. https://doi.org/10.1016/j.system.2018.03.003.

DOI: https://doi.org/10.21831/lt.v11i2.67170