Item parameters of Yureka Education Center (YEC) English Proficiency Online Test (EPOT) instrument

Endrati Jati Siwi; Rosyita Anindyarini; Sabiqun Nahar

doi:10.21831/reid.v6i1.31013

Item parameters of Yureka Education Center (YEC) English Proficiency Online Test (EPOT) instrument

Endrati Jati Siwi, Yureka Education Center, Yogyakarta, Indonesia
Rosyita Anindyarini, Yureka Education Center, Yogyakarta, Indonesia
Sabiqun Nahar, Yureka Education Center, Yogyakarta, Indonesia

10.21831/reid.v6i1.31013

Abstract

Yureka Education Center (YEC) is one of the institutions which has developed an online-based English proficiency test. The test is called the English Proficiency Online Test (EPOT) which follows the TOEFL ITP (Institutional Testing Program) framework. Thus, this study aimed to analyze the characteristics of EPOT instruments consisting of Listening, Structure, and Reading subtests, which later the quality of each EPOT test item is identified. This study used a descriptive quantitative approach by describing the characteristics of EPOT test items in terms of item difficulty index, item discrimination index, test information’s function, and test measurement’s errors. The data were collected through EPOT trials conducted by 2,652 online test-takers as participants from 20 provinces in Indonesia. The collected data were then analyzed using the Item Response Theory (IRT) approach using the BILOG program on all logistic parameter models which began with the item compatibility test against the model. Based on the results of the analysis, all subtests match the 3-PL model. Most of EPOT’s test items had a good range of difficulty index and discrimination index. The EPOT information’s function shows that accurate items are used on the 3-PL model for a certain capability range. This study is expected to point out that the EPOT test could be used as an alternative English proficiency test that is easy to use and useful.

Keywords

analysis; parameter; EPOT; listening; structure; reading

Full Text:

PDF

References

Anderson, P., & Morgan, G. (2008). Developing tests and questionnaires for a national assessment of educational achievement (V. Greaney & T. Kellaghan, Eds.). https://doi.org/10.1596/978-0-8213-7497-9

Arnani, M. (2019, November 14). CPNS 2019, 9 instansi ini wajibkan TOEFL, berapa skornya? Kompas.Com. Retrieved from https://www.kompas.com/tren/read/2019/11/14/120925265/cpns-2019-9-instansi-ini-wajibkan-toefl-berapa-skornya?page=all

Azwar, S. (2017). Reliabilitas dan validitas (4th ed.). Yogyakarta: Pustaka Pelajar.

Chung, H. (2005). Calibration and validation of the body self-image questionnaire using the Rasch analysis. Master thesis, University of Georgia, Athens, GA.

Crocker, L. M., & Algina, J. (1986). Introduction to classical and modern test theory. Fort Worth, TX: Harcourt Brace Jovanovich.

de Ayala, R. J. (2009). The theory and practice of Item Response Theory. New York, NY: Guilford Press.

Fan, X. (1998). Item Response Theory and Classical Test Theory: An empirical comparison of their item/person statistics. Educational and Psychological Measurement, 58(3), 357–381. https://doi.org/10.1177/0013164498058003001

Gliem, J. A., & Gliem, R. R. (2003). Calculating, interpreting, and reporting Cronbach’s Alpha reliability coefficient for Likert-type scales. Midwest Research-to-Practice Conference in Adult, Continuing, and Community Education, 82–88. Colombus, OH: The Ohio University.

Guilford, J. P. (1956). Fundamental statistics in psychology and education (3rd ed.). New York, NY: McGraw-Hill.

Hambleton, R. K. (1989). Principles and selected applications of item response theory. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 147–200). New York, NY: Macmillan.

Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park, CA: Sage Publications.

Handayani, S. (2016). Pentingnya kemampuan Bahasa Inggris dalam menyongsong ASEAN Community 2015. Jurnal Profesi Pendidik, 3(1), 102–106. Retrieved from http://ispijateng.org/wp-content/uploads/2016/05/PENTINGNYA-KEMAMPUAN-BERBAHASA-INGGRIS-SEBAGAI-DALAM-MENYONGSONG-ASEAN-COMMUNITY-2015_Sri-Handayani.pdf

Hingorjo, M. R., & Jaleel, F. (2012). Analysis of one-best MCQs: The difficulty index, discrimination index and distractor efficiency. JPMA: The Journal of the Pakistan Medical Association, 62(2), 142–147. Retrieved from https://jpma.org.pk/article-details/3255?article_id=3255

Hulin, C. L., Drasgow, F., & Parsons, C. K. (1983). Item response theory: Application to psychological measurement. Homewood, IL: Dow Jones- Irwin.

Huriaty, D. (2019). Analisis karakteristik parameter butir berdasarkan model Logistik 3 Parameter. Lentera: Jurnal Pendidikan, 14(2), 33–40. https://doi.org/10.33654/jpl.v14i2.885

Magis, D. (2013). A note on the item information function of the four-parameter logistic model. Applied Psychological Measurement, 37(4), 304–315. https://doi.org/10.1177/0146621613475471

Naga, D. S. (1992). Pengantar teori sekor pada pengukuran pendidikan. Jakarta: Gunadarma.

Nelson, L. (2001). Item analysis for test and surveys using Lertap 5. Perth: Curtin University of Technology.

Olufemi, A. S. (2013). Item Response Theory as a basis for measuring latent trait of interest. Greener Journal of Social Sciences, 3(7), 378–382. https://doi.org/10.15580/GJSS.2013.7.062513691

Pollard, B., Dixon, D., Dieppe, P., & Johnston, M. (2009). Measuring the ICF components of impairment, activity limitation and participation restriction: An item analysis using classical test theory and item response theory. Health and Quality of Life Outcomes, 7, 1–20. https://doi.org/10.1186/1477-7525-7-41

Retnawati, H. (2014). Teori respons butir dan penerapannya: Untuk peneliti, praktisi pengukuran dan pengujian, mahasiswa pascasarjana. Yogyakarta: Nuha Medika.

Setiawati, F. A., Izzaty, R. E., & Hidayat, V. (2018). Analisis respons butir pada tes bakat skolastik. Jurnal Psikologi, 17(1), 1–17. https://doi.org/10.14710/jp.17.1.1-17

Sharpe, P. J. (2002). How to prepare for the TOEFL test: Test of English as a foreign language (10th ed.). Jakarta: Binarupa Aksara.

Sugiyono, S. (2014). Metode penelitian pendidikan: Pendekatan kuantitatif, kualitatif, dan R&D. Bandung: Alfabeta.

van der Linden, W. J., & Hambleton, R. K. (1996). Handbook of modern item response theory. https://doi.org/10.1007/978-1-4757-2691-6 I

Wells, C. S., & Wollack, J. A. (2003). An instructor’s guide to understanding test reliability. Madison, WI: University of Wisconsin.

Yang, F. M., & Kao, S. T. (2014). Item response theory for measurement validity. Shanghai Archives of Psychiatry, 26(3), 171–177. https://doi.org/10.3969/j.issn.1002-0829.2014.03

Yen, W., & Fitzpatrick, A. (2006). Item response theory. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 111–153). Westport, CT: American Council on Education and Praeger.

Zięba, A. (2013). The item information function in one and two-parameter logistic models – A comparison and use in the analysis of the results of school tests. Didactics of Mathematics, 10(14), 87–96. https://doi.org/10.15611/dm.2013.10.08

DOI: https://doi.org/10.21831/reid.v6i1.31013