The application of the Generalized Lord’s Chi-Square method in identifying biased items

Herwin Herwin; Sophak Phonn

doi:10.21831/pep.v23i1.20665

The application of the Generalized Lord’s Chi-Square method in identifying biased items

Herwin Herwin, Faculty of Education, Universitas Negeri Yogyakarta, Indonesia
Sophak Phonn, Special Education Department, Ministry of Education, Youth and Sport of Cambodia, Cambodia

10.21831/pep.v23i1.20665

Abstract

This study aims to find out the results of the analysis of item bias by using Generalized Lord's Chi-square test method on the test instrument of elementary school examination in a sub-district of Gowa Regency, Indonesia. This research is explorative research using quantitative approach. This research was conducted in the second semester of the academic year of 2017/2018 at the Bontomarannu District Elementary School in Gowa Regency. Data collection technique used is documentation. The data in this study were analyzed using the DIF method of Generalized Lord Chi-Square test. The results show that in using the Generalized Lord Chi-square method, from 20 items of mathematics test of school exam in Bontomarannu District Elementary School in Gowa Regency of the academic year 2017/2018, there are two items which are detected containing the bias (DIF), i.e., item 5 and item 11, while the rest are not.

Keywords

DIF; generalized Lord’s chi-square; school examination

Full Text:

PDF

References

Adams, R. J. (1992). Item bias. In J. P. Keeves (Ed.), The IEA technical handbook. The Hague: The International Association for the Evaluation of Educational Achiement (IEA).

Bares, C., Andrade, F., Delva, J., Grogan-Kaylor, A., & Kamata, A. (2012). Differential item functioning due to gender between depression and anxiety items among Chilean adolescents. International Journal of Social Psychiatry, 58(4), 386–392. https://doi.org/10.1177/0020764011400999

Budiyono, B. (2009). Ketepatan metode Mantel-Haenszal, Sibtest, dan Regresi Logistik untuk mendeteksi differential item function. Jurnal Penelitian Dan Evaluasi Pendidikan, 13(1), 1–20. https://doi.org/10.21831/pep.v13i1.1398

Chiesi, F., Ciancaleoni, M., Galli, S., Morsanyi, K., & Primi, C. (2012). Item response theory analysis and differential item functioning across age, gender and country of a short form of the Advanced Progressive Matrices. Learning and Individual Differences, 22(3), 390–396. https://doi.org/10.1016/j.lindif.2011.12.007

Cho, Y. I., Martin, M. J., Conger, R. D., & Widaman, K. F. (2010). Differential item functioning on antisocial behavior scale items for adolescents and young adults from single-parent and two-parent families. Journal of Psychopathology and Behavioral Assessment, 32(2), 157–168. https://doi.org/10.1007/s10862-009-9145-1

Colom, R., & Garcı́a-López, O. (2002). Sex differences in fluid intelligence among high school graduates. Personality and Individual Differences, 32(3), 445–451. https://doi.org/10.1016/S0191-8869(01)00040-X

De Leo, J. A., Van Dam, N. T., Hobkirk, A. L., & Earleywine, M. (2011). Examining bias in the impulsive sensation seeking (ImpSS) Scale using Differential Item Functioning (DIF) – An item response analysis. Personality and Individual Differences, 50(5), 570–576. https://doi.org/10.1016/j.paid.2010.11.030

Herwin, H. (2016). An application of the generalized logistic regression method in identifying DIF. In Proceeding of the International Conference on Educational Research and Evaluation (ICERE). Yogyakarta: Universitas Negeri Yogyakarta.

Jodoin, M. G., & Gierl, M. J. (2001). Evaluating type I error and power rates using an effect size measure with the logistic regression procedure for DIF detection. Applied Measurement in Education, 14(4), 329–349. https://doi.org/10.1207/S15324818AME1404_2

Karami, H. (2012). An introduction to differential item functioning. The International Journal of Educational and Psychological Assessment, 11(2), 59–76.

Kim, S.-H., Cohen, A. S., & Park, T.-H. (1995). Detection of differential item functioning in multiple groups. Journal of Educational Measurement, 32(3), 261–276. https://doi.org/10.2307/1435297

Lautamo, T., Laakso, M.-L., Aro, T., Ahonen, T., & Törmäkangas, K. (2011). Validity of the play assessment for group settings: An evaluation of differential item functioning between children with specific language impairment and typically developing peers. Australian Occupational Therapy Journal, 58(4), 222–230. https://doi.org/10.1111/j.1440-1630.2011.00941.x

Magis, D., Béland, S., Tuerlinckx, F., & De Boeck, P. (2010). A general framework and an R package for the detection of dichotomous differential item functioning. Behavior Research Methods, 42(3), 847–862. https://doi.org/10.3758/BRM.42.3.847

Osterlind, S. J. (1983). Test item bias. Series: Quantitative applications in the social sciences. Beverly Hills, CA: Sage.

Retnawati, H. (2013). Pendeteksian keberfungsian butir pembeda dengan indeks volume sederhana berdasarkan teori respons butir multidimensi. Jurnal Penelitian Dan Evaluasi Pendidikan, 17(2), 275–286. https://doi.org/10.21831/pep.v17i2.1700

Retnawati, H. (2014). Teori respons butir dan penerapannya: Untuk peneliti, praktisi pengukuran dan pengujian, mahasiswa pascasarjana. Yogyakarta: Nuha Medika.

Scheuneman, J. D., & Bleistein, C. A. (1999). Item bias. In G. N. Masters & J. P. Keeves (Eds.), Advances in Measurement in Educational Research and Assessment (pp. 220–234). New York, NY: Elsevier.

Weisscher, N., Glas, C. A., Vermeulen, M., & De Haan, R. J. (2010). The use of an item response theory-based disability item bank across diseases: Accounting for differential item functioning. Journal of Clinical Epidemiology, 63(5), 543–549. https://doi.org/10.1016/j.jclinepi.2009.07.016

Zhang, Y. (2015). Multiple ways to detect differential item functioning in SAS. In Proceedings of SAS Global Forum 2015 Conference (pp. 1–9). Dallas, TX: SAS Global Forum. Retrieved from https://pdfs.semanticscholar.org/01f8/0a01287893f8f3f1029aa817b9cce3983901.pdf

Zumbo, B. D. (1999). A handbook on the theory and methods of differential item functioning (DIF). Ottawa, Canada: Directorate of Human Resources Research and Evaluation.

DOI: https://doi.org/10.21831/pep.v23i1.20665