Quality analysis of teacher-made tests in financial accounting subject at vocational high schools

Heni Mulyani; Heraeni Tanuatmodjo; Rangga Iskandar

doi:10.21831/jpv.v10i1.29382

Quality analysis of teacher-made tests in financial accounting subject at vocational high schools

Heni Mulyani, Universitas Pendidikan Indonesia, Indonesia
Heraeni Tanuatmodjo, Universitas Pendidikan Indonesia, Indonesia
Rangga Iskandar, Universitas Pendidikan Indonesia, Indonesia

10.21831/jpv.v10i1.29382

Abstract

Assessment of student learning outcomes needs to be done using tests that meet the criteria for quality tests. This study aims to determine the quality of teacher-made tests on financial accounting subjects in Vocational High Schools. This research is descriptive research with a quantitative approach. Data collected are questions made by 32 teachers, answer sheets from 689 Accounting students. The validity of objective and essay tests using product-moment correlation. Reliability of the objective test using the KR20 formula, while the essay test using Alpha formula. Difficulty level and distinguishing power of objective tests using Anates 4. Difficulty level and distinguishing power of essay tests used Microsoft Excel 2013. The research results obtained are as follows: (1) validity of teacher-made test items cannot accurately measure learning outcomes; (2) reliability of teacher-made tests cannot show stable results despite repeated testing of the same subject; (3) teacher-made tests do not have a proportion of degree of difficulty that is suitable for use as a Mid-Semester assessment tool; (4) distinguishing power of tests made by teachers cannot distinguish students who have mastered the test material (upper or superior group) from students who have not mastered the test material (lower group or user); (5) Multiple choice test distractors made by teachers are not evenly chosen, and the key options and deception options do not function effectively. Quality analysis of teacher-made tests through item analysis is intended to identify damaged test items and to show areas that are already mastered by students.

Keywords

teacher-made test; test quality; financial accounting; reliability; validity

Full Text:

PDF

References

Arikunto, S. (2012). Dasar-dasar evaluasi pendidikan. Bumi Aksara.

Ashtiani, N. S., & Babaii, E. (2007). Cooperative test construction: The last temptation of educational reform? Studies in Educational Evaluation, 33(3–4), 213–228. https://doi.org/10.1016/j.stueduc.2007.07.002

Bichi, A. A. (2015). Item analysis using a derived science achievement test data. International Journal of Science and Research (IJSR), 4(5), 1656–1662.

Black, P., Harrison, C., Hodgen, J., Marshall, B., & Serret, N. (2010). Validity in teachers’ summative assessments. Assessment in Education: Principles, Policy & Practice, 17(2), 215–232. https://doi.org/10.1080/09695941003696016

Carroll, T., & Moody, L. (2006). Teacher-made tests. Science Scope, September, 66. https://www.questia.com/library/journal/1G1-153041130/teacher-made-tests

Clark, I. (2008). Assessment is for learning: Formative assessment and positive learning interactions. Florida Journal of Educational Administration & Policy, 2(1), 1–16.

Cooper, T., Ashley, P., & Simona, W. (2014). Using reliability, validity, and item analysis to evaluate a teacher-developed test in international business. Evaluation and Testing Research Article, 1–11.

DiDonato-Barnes, N., Fives, H., & Krause, E. S. (2014). Using a table of specifications to improve teacher-constructed traditional tests: an experimental design. Assessment in Education: Principles, Policy & Practice, 21(1), 90–108.

Frisbie, D. A. (1988). Reliability of scores from teacher‐made tests. Educational Measurement: Issues and Practice, 7(1), 25–35.

Grant, L., & Gareis, C. (2015). Teacher-made assessments: How to connect curriculum, instruction, and student learning. Routledge.

Hamzah, M. S. G., & Abdullah, S. K. (2011). Test item analysis: An educator professionalism approach. US-China Education Review, A(3), 307–322.

Heyneman, S. P., & Fägerlind, I. (1988). University examinations and standardized testing: Principles, experience, and policy options. World Bank.

Ing, L. M., Musah, M. B., Al-Hudawi, S. H., Tahir, L. M., & Kamil, N. M. (2015). Validity of teacher-made assessment: A table of specification approach. Asian Social Science, 11(5). https://doi.org/10.5539/ass.v11n5p193

Khairani, A. Z., & Shamsuddin, H. (2016). Assessing item difficulty and discrimination indices of teacher-developed multiple-choice tests. In Assessment for learning within and beyond the classroom (pp. 417–426). Springer Singapore. https://doi.org/10.1007/978-981-10-0908-2_35

Kinyua, K., & Okunya, L. O. (2014). Validity and reliability of teacher-made tests: Case study of year 11 physics in Nyahururu district of Kenya. African Educational Research Journal, 2(2), 61–71.

Kusaeri, K., & Suprananto, S. (2012). Pengukuran dan penilaian pendidikan. Graha Imu.

Lee, J., & Lee, Y. S. (2013). Effects of testing. In J. Hattie & E. M. Anderman (Eds.), International guide to student achievement (pp. 416–418). Routledge.

Levy, P., & Goldstein, H. (2014). Tests in education: A book of critical reviews. Academic Press.

Linn, R. L., & Gronlund, N. (2000). Measurement and assessment in teaching (8th ed.). Prentice-Hall.

Marso, R. N., & Pigge, F. L. (1991). An analysis of teacher-made tests: Testing practices, cognitive demands and item construction errors. Contemporary Educational Psychology, 16, 279–286.

Meshkani, Z., & Abadie, F. H. (2005). Multivariate analysis of factors influencing reliability of teacher made tests. Journal of Medical Education, 6(2), e105155. https://doi.org/10.22037/jme.v6i2.765

Mitra, N. K., Nagaraja, H. S., Ponnudurai, G., & Judson, J. P. (2009). The levels of difficulty and discrimination indices in type A multiple choice questions of pre-clinical semester 1 multidisciplinary summative tests. IeJSME, 3(1), 2–7.

Nordin, M. S. (2002). Testing and interpreting in classrooms. International Islamic University of Malaysia Publications.

Notar, C. E., Zuelke, D. C., Wilson, J. D., & Yunker, B. D. (2004). The table of specifications: Insuring accountability in teacher made tests. Journal of Instructional Psychology, 31(2). https://www.questia.com/library/journal/1G1-119611686/the-table-of-specifications-insuring-accountability

Parkes, J. (2013). Reliability in classroom assessment. In J. H. McMillan (Ed.), SAGE handbook of research on classroom assessment (pp. 107–123). SAGE Publications.

Popham, W. J. (2009). Assessment literacy for teachers: Faddish or fundamental? Theory Into Practice, 48(1), 4–11. https://doi.org/10.1080/00405840802577536

Quaigrain, K., & Arhin, A. K. (2017). Using reliability and item analysis to evaluate a teacher-developed test in educational measurement and evaluation. Cogent Education, 4(1), 1301013. https://doi.org/10.1080/2331186X.2017.1301013

Reynolds, C. R., Livingston, R. B., & Willson, V. (2010). Measurement and assessment in education (2nd ed.). Pearson Education.

Sudijono, A. (2015). Pengantar evaluasi pendidikan. Raja Grafindo Persada.

Walker, C. M., Schmidt, E., & Mototsune, K. (2004). Smart tests: Teacher-made tests that help students learn. Pembroke Publishers Limited.

Wiggins, G. (1989). A true test: Toward more authentic and equitable assessment. Phi Delta Kappan, 70, 703–710.

Winter, P. C., Kopriva, R. J., Chen, C.-S., & Emick, J. E. (2006). Exploring individual and item factors that affect assessment validity for diverse learners: Results from a large-scale cognitive lab. Learning and Individual Differences, 16(4), 267–276. https://doi.org/10.1016/j.lindif.2007.01.001

Wright, R. J. (2007). Educational assessment: Tests and measurements in the age of accountability. Sage Publications.

Young, V. M., & Kim, D. H. (2010). Using assessments for instructional improvement: A literature review. Education Policy Analysis Archives, 18(19), 1–40. https://doi.org/10.14507/epaa.v18n19.2010

DOI: https://doi.org/10.21831/jpv.v10i1.29382