Gender differential item functioning on the Kentucky Inventory of Mindfulness Skills instrument using logistic regression

Sumin Sumin; Fitri Sukmawati; Nurdin Nurdin

doi:10.21831/reid.v8i1.50809

Gender differential item functioning on the Kentucky Inventory of Mindfulness Skills instrument using logistic regression

Sumin Sumin, Institut Agama Islam Negeri Pontianak, Indonesia
Fitri Sukmawati, Institut Agama Islam Negeri Pontianak, Indonesia
Nurdin Nurdin, Dinas Pendidikan Provinsi Aceh, Indonesia

10.21831/reid.v8i1.50809

Abstract

The item differential function (DIF) describes a situation in which testees of similar ability but from different demographic groups have varying chances of achieving the same result. This study aims to identify the function of uniform and non-uniform differential items on the Kentucky Inventory of Mindfulness Skills Instrument using logistic regression techniques and determine the impact of DHF on construct validity. This study uses a survey method with a quantitative approach. The study involved 602 people, divided into two groups based on gender: 301 women and 301 men. The Kentucky Inventory of Mindfulness Skills (KIMS) is a 39-item online questionnaire that measures mindfulness. KIMS has been proven to meet content, construct, and factor validity and has good test-retest reliability and internal consistency estimators. This study uses Regression Logistics to detect DIF, analyzed with R Studio 4.1.3 software. Research results found 17 DIF items detected using logistic regression, 13 uniform DIF items, and four non-uniform DIF. Through CFA, we have succeeded in proving that DIF-free items are proven to have better construct validity. The implications of this study are expected to inspire counseling psychologists to be more careful in using rating scales or instruments. The validity and reliability of the measures are not strong enough to justify that all measuring instruments are correct. However, it is also necessary to check for item bias or functional differential items to ensure that each item on the scale or instrument is understandable to all demographic groups and does not benefit only certain demographic groups.

Keywords

item biases; differential item functioning; logistic regression

Full Text:

PDF

References

Abedalaziz, N. (2010). Detecting gender related DIF using logistic regression and Mantel-Haenszel approaches. Procedia-Social and Behavioral Sciences, 7, 406–413. https://doi.org/10.1016/j.sbspro.2010.10.055.

Abedalaziz, N., Leng, Â. C. H., & Alahmadi, Â. A. (2018). Detecting a gender-related differential item functioning using transformed item difficulty. MOJES: Malaysian Online Journal of Educational Sciences, 2(1), 16–22. https://ejournal.um.edu.my/index.php/MOJES/article/view/12820.

Baer, R. A., Smith, G. T., & Allen, K. B. (2004). Assessment of mindfulness by self-report: The Kentucky inventory of mindfulness skills. Assessment, 11(3), 191–206. https://doi.org/10.1177/1073191104268029.

Bandalos, D. L. (2018). Measurement theory and applications for the social sciences. Guilford Publications.

Camilli, G., & Congdon, P. (1999). Application of a method of estimating DIF for polytomous test items. Journal of Educational and Behavioral Statistics, 24(4), 323–341. https://doi.org/10.3102/10769986024004323.

Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81-105. https://doi.org/10.1037/h0046016.

Chen, H.-F., & Jin, K.-Y. (2018). Applying logistic regression to detect Differential Item Functioning in multidimensional data. Frontiers in Psychology, 9, 1302. https://doi.org/10.3389/fpsyg.2018.01302.

Creswell, J. W., & Poth, C. N. (2016). Qualitative inquiry and research design: Choosing among five approaches. Sage publications.

Cuevas, M., & Cervantes, V. H. (2012). Differential item functioning detection with logistic regression. Psychologie et Mathématiques, 3, 45-59. https://doi.org/10.4000/msh.12274.

Desjardins, C. D., & Bulut, O. (2018). Handbook of educational measurement and psychometrics using R. CRC Press.

Fornell, C., & Larcker, D. F. (1981). Structural equation models with unobservable variables and measurement error: Algebra and statistics. Sage Publications.

Ghazali, I. (2014). SEM metode alternatif dengan menggunakan Partial Least Squares (PLS). Universitas Diponegoro Semarang.

Huang, T. -W., Wu, P. -C., & Mok, M. M. C. (2022). Examination of gender-related differential item functioning through Poly-BW indices. Frontiers in Psychology, 13, 821459. https://doi.org/10.3389/fpsyg.2022.821459.

Jodoin, M. G., & Gierl, M. J. (2001). Evaluating type I error and power rates using an effect size measure with the logistic regression procedure for DIF detection. Applied Measurement in Education, 14(4), 329–349. https://doi.org/10.1207/S15324818AME1404_2.

Jöreskog, K. G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34(2), 183–202. https://doi.org/10.1007/BF02289343.

Lee, S. (2017). Detecting differential item functioning using the logistic regression procedure in small samples. Applied Psychological Measurement, 41(1), 30–43. https://doi.org/10.1177/0146621616668015.

Martinková, P., Drabinová, A., Liaw, Y.-L., Sanders, E. A., McFarland, J. L., & Price, R. M. (2017). Checking equity: Why differential item functioning analysis should be a routine part of developing conceptual assessments. CBE—Life Sciences Education, 16(2), rm2. https://doi.org/10.1187/cbe.16-10-0307.

Mazor, K. M., Kanjee, A., & Clauser, B. E. (1995). Using logistic regression and the Mantel‐Haenszel with multiple ability estimates to detect differential item functioning. Journal of Educational Measurement, 32(2), 131–144. https://doi.org/10.1111/j.1745-3984.1995.tb00459.x.

Özdemir, B. (2015). A comparison of IRT-based methods for examining differential item functioning in TIMSS 2011 mathematics subtest. Procedia-Social and Behavioral Sciences, 174, 2075–2083. https://doi.org/10.1016/j.sbspro.2015.02.004.

Özgümüs, A., Rau, H. A., Trautmann, S. T., & König-Kersting, C. (2020). Gender Bias in the Evaluation of Teaching Materials. Frontiers in Psychology, 11. https://doi.org/10.3389/fpsyg.2020.01074.

Saygin, P. O., & Atwater, A. (2021). Gender differences in leaving questions blank on high-stakes standardized tests. Economics of Education Review, 84, 102162. https://doi.org/10.1016/j.econedurev.2021.102162.

Stark, S., Chernyshenko, O. S., & Drasgow, F. (2006). Detecting differential item functioning with confirmatory factor analysis and item response theory: Toward a unified strategy. Journal of Applied Psychology, 91(6), 1292-1306. https://doi.org/10.1037/0021-9010.91.6.1292.

Swaminathan, H., & Rogers, H. J. (1990). Detecting Differential Item Functioning using logistic regression procedures. Journal of Educational Measurement, 27(4), 361–370. http://www.jstor.org/stable/1434855.

Wiberg, M. (2007). Measuring and detecting differential item functioning in criterion-referenced licensing test: A theoretic comparison of methods. Institutionen för beteendevetenskapliga mätningar, Umeå.

Zumbo, B. D. (1999). A handbook on the theory and methods of differential item functioning (DIF). National Defense Headquarters.

DOI: https://doi.org/10.21831/reid.v8i1.50809