Item quality analysis using the Rasch model to measure critical thinking ability in the material of the human digestive system of Biology subject in high school

Wanda Agus Prasetya; Anggi Tias Pratama

doi:10.21831/pep.v27i1.58873

Item quality analysis using the Rasch model to measure critical thinking ability in the material of the human digestive system of Biology subject in high school

Wanda Agus Prasetya, Universitas Negeri Yogyakarta, Indonesia
Anggi Tias Pratama, Universitas Negeri Yogyakarta, Indonesia

10.21831/pep.v27i1.58873

Abstract

This study aims to determine the quality of the Biology instrument items on the digestive system material with the Rasch model for analyzing critical thinking skills. The research employed a quantitative descriptive method involving 63 students of senior high schools in Yogyakarta. The data were collected using a critical thinking skills description test and processed using the Rasch model with the Winstep 5.0.3.4 program. The study shows that the overall validity is acceptable. The item validity did not require improvement in items 14, 3, 8, 13, 12, 1, 4, 10, 6, 2, 7, 11, 15, and 9, and required improvement or replaced of item 5 because it did not fit. The analysis result using Cronbach's alpha shows that the overall reliability is very good, and the item reliability is good. Rating scale analysis using partial credit ratings and probability curves shows that respondents need help understanding the five-point Likert scale. The analysis of the item difficulty based on Logit and Wright maps shows that the most difficult item to work on is item 14. Items with moderate categories are items 13, 12, 1, 4, 10, 6, 2, 5, and 7. Items easy to work on are items 11, 15, and 9. The bias results show item 14 gender-biased. The results of the interaction between the item and the person through the ICC plot image show that all items are on the curve of the outfit confidence space and follow the Rasch modeling.

Keywords

Rasch model; critical thinking ability; human digestive system

Full Text:

PDF

References

AERA & APA. (2014). Standards for educational and psychological testing. American Educational Research Association.

Agustina, D. F., Raharjo, R., Isnawati, I., & Hartono, D. (2023). Test instrument based on critical thinking skills integrated Javanese cultural tradition in Islamic context. International Journal of Social Science And Human Research, 6(2), 987-995. https://doi.org/10.47191/ijsshr/v6-i2-30

Andrich, D. (2013a). An expanded derivation of the threshold structure of the polytomous Rasch model that dispels any “Threshold Disorder Controversy.” Educational and Psychological Measurement, 73(1), 78–124. https://doi.org/10.1177/0013164412450877

Andrich, D. (2013b). The legacies of R. A. Fisher and K. Pearson in the application of the polytomous Rasch model for assessing the empirical ordering of categories. Educational and Psychological Measurement, 73(4), 553–580. https://doi.org/10.1177/0013164413477107

Ardianto, D., Rubini, B., & Pursitasari, I. D. (2023). Assessing STEM career interest among secondary students: A Rasch model measurement analysis. Eurasia Journal of Mathematics, Science and Technology Education, 19(1), em2213. https://doi.org/10.29333/ejmste/12796

Ariesta, P., Susanti, R., & Rahayu, E. S. (2019). The influence of Conceptual Understanding Procedures (CuPs) learning model with (the use of) Bio-Quartet cards. J.Biol.Educ., 8(1), 50–55. http://journal.unnes.ac.id/sju/index.php/ujbe

Austvoll-Dahlgren, A., Guttersrud, Ø., Nsangi, A., Semakula, D., & Oxman, A. D. (2017). Measuring ability to assess claims about treatment effects: A latent trait analysis of items from the “Claim Evaluation Tools” database using Rasch modelling. BMJ Open, 7(5), e013185. https://doi.org/10.1136/bmjopen-2016-013185

Azizah, N., Suseno, M., & Hayat, B. (2022). Item analysis of the Rasch model items in the final semester exam indonesian language lesson. World Journal of English Language, 12(1), 15–26. https://doi.org/10.5430/wjel.v12n1p15

Basri, H., Purwanto, P., As’ari, A. R., & Sisworo, S. (2019). Investigating critical thinking skill of junior high school in solving mathematical problem. International Journal of Instruction, 12(3), 745–758. https://doi.org/10.29333/iji.2019.12345a

Benson, N. F., Beaujean, A. A., Donohue, A., & Ward, E. (2018). W Scores: Background and derivation. Journal of Psychoeducational Assessment, 36(3), 273–277. https://doi.org/10.1177/0734282916677433

Bonsaksen, T., Kottorp, A., Gay, C., Fagermoen, M. S., & Lerdal, A. (2013). Rasch analysis of the general self-efficacy scale in a sample of persons with morbid obesity. Health and Quality of Life Outcomes, 11, 202 https://doi.org/10.1186/1477-7525-11-202

Boone, W. J., Staver, J. R., & Yale, M. S. (2014). Rasch analysis in the human sciences. Springer Netherlands. https://doi.org/10.1007/978-94-007-6857-4

Cantó-Cerdán, M., Cacho-Martínez, P., Lara-Lacárcel, F., & García-Muñoz, Á. (2021). Rasch analysis for development and reduction of Symptom Questionnaire for Visual Dysfunctions (SQVD). Scientific Reports, 11(1), 14855. https://doi.org/10.1038/s41598-021-94166-9

Center for Educational Assessment. (2018). Pendidikan di Indonesia: Belajar dari hasil PISA 2018 programme for international student assessment. Center for Educational Assessment, Badan Research and Development Agency, Ministry of Education and Culture.

Chan, S. W., Ismail, Z., & Sumintono, B. (2014). A Rasch model analysis on secondary students’ statistical reasoning ability in descriptive statistics. Procedia - Social and Behavioral Sciences, 129, 133–139. https://doi.org/10.1016/j.sbspro.2014.03.658

Chukwuyenum, A. N. (2013). Impact of critical thinking on performance in Mathematics among senior secondary school students in Lagos State. IOSR Journal of Research & Method in Education, 3(5), 27910355. https://doi.org/10.9790/7388-0351825

Danczak, S. M., Thompson, C. D., & Overton, T. L. (2017). What does the term critical thinking mean to you? A qualitative analysis of chemistry undergraduate, teaching staff and employers’ views of critical thinking. Chemistry Education Research and Practice, 18(3), 420–434. https://doi.org/10.1039/c6rp00249h

Facione, P. A. (1992). Critical thinking: What it is and why it counts. Insight Assessment.

Faradillah, A., & Adlina, S. (2021). Validity of critical thinking skills instrument on prospective Mathematics teachers. Jurnal Penelitian dan Evaluasi Pendidikan, 25(2), 126-137. https://doi.org/10.21831/pep.v25i2.40662

Claro, H. C., de Oliveira, M. A. F., Fernandes, I. F. A. L., Titus, J. C., Tarifa, R. R., Rojas, T. F., & Pinho, P. H. (2015). Rasch model of the GAIN substance problem scale among inpatient and outpatient clients in the city of São Paulo, Brazil. Addictive Behaviors Reports, 2, 55–60. https://doi.org/10.1016/j.abrep.2015.08.001

Göçmen, Ö., & Coşkun, H. (2019). The effects of the six thinking hats and speed on creativity in brainstorming. Thinking Skills and Creativity, 31, 284–295. https://doi.org/10.1016/j.tsc.2019.02.006

Hamdu, G., Fuadi, F. N., Yulianto, A., & Akhirani, Y. S. (2020). Items quality analysis using Rasch model to measure elementary school students’ critical thinking skill on Stem learning. JPI (Jurnal Pendidikan Indonesia), 9(1), 61-74. https://doi.org/10.23887/jpi-undiksha.v9i1.20884

Hansen, T., & Kjaersgaard, A. (2020). Item analysis of the Eating Assessment Tool (EAT-10) by the Rasch model: A secondary analysis of cross-sectional survey data obtained among community-dwelling elders. Health and Quality of Life Outcomes, 18(1), 1–14. https://doi.org/10.1186/s12955-020-01384-2

Hasanah, S. N., Sunarno, W., & Prayitno, B. A. (2020). Profile of students’ critical thinking skills in junior high schools in Surakarta. In Proceedings of the 3rd International Conference on Learning Innovation and Quality Education (ICLIQE 2019), pp. 570-575. https://doi.org/10.2991/assehr.k.200129.070

Imani, V., Lin, C. Y., Jalilolghadr, S., & Pakpour, A. H. (2018). Factor structure and psychometric properties of a Persian translation of the Epworth Sleepiness Scale for children and adolescents. Health Promotion Perspectives, 8(3), 200–207. https://doi.org/10.15171/hpp.2018.27

Karoror, I., & Jalmo, T. (2022). Profile of critical thinking ability in Ecosystem materials using the Rasch model. Jurnal Penelitian Pendidikan IPA, 3(8), 1599-1604. https://doi.org/10.29303/jppipa.v8i3.1394

Kartimi, K. (2012). Pengembangan alat ukur berpikir kritis pada konsep Termokimia untuk siswa SMA. Jurnal Scientiae Educatia, 1(1), 1-14. https://www.syekhnurjati.ac.id/jurnal/index.php/sceducatia/article/view/501

Khine, M. S. (2020). Rasch measurement: Applications in quantitative educational research. In Rasch measurement: Applications in quantitative educational research. Springer Singapore. https://doi.org/10.1007/978-981-15-1800-3

Kim, J. (2021). Development and validation of the career adaptability scale for undergraduates in Korea. Sustainability (Switzerland), 13(19), 11004. https://doi.org/10.3390/su131911004

Lin, C. Y., Broström, A., Nilsen, P., Griffiths, M. D., & Pakpour, A. H. (2017). Psychometric validation of the Persian bergen social media addiction scale using classic test theory and Rasch models. Journal of Behavioral Addictions, 6(4), 620–629. https://doi.org/10.1556/2006.6.2017.071

Linacre, J. M. (2002). Optimizing rating scale category effectiveness. Journal of Applied Measurement, 3(1), 85-106. https://europepmc.org/article/med/11997586

Madyani, I., Yamtinah, S., Utomo, S. B., Saputro, S., & Mahardiani, L. (2020). Profile of students’ creative thinking skills in science learning. In Proceedings of the 3rd International Conference on Learning Innovation and Quality Education (ICLIQE 2019), pp. 957-964. https://doi.org/10.2991/assehr.k.200129.119

Matondang, Z. (2009). Validitas dan reliabilitas suatu instrumen penelitian. Jurnal Tabularasa PPs Unimed, 6(1), 87-97. http://digilib.unimed.ac.id/705/1/Validitas%20dan%20reliabilitas%20suatu%20instrumen%20penelitian.pdf

McAlinden, C., Khadka, J., Santos Paranhos, J. de F., Schor, P., & Pesudovs, K. (2012). Psychometric properties of the NEI-RQL-42 questionnaire in keratoconus. Investigative Ophthalmology and Visual Science, 53(11), 7370–7374. https://doi.org/10.1167/iovs.12-9969

McCamey, R. (2014). A primer on the one-parameter Rasch model. American Journal of Economics and Business Administration, 6(4), 159–163. https://doi.org/10.3844/ajebasp.2014.159.163

Miarti, E., Hasnunidah, N., & Abdurrahman, A. (2021). The effect of learning cycle 5E on critical thinking skills for junior high school students. Scientiae Educatia, 10(2), 177. https://doi.org/10.24235/sc.educatia.v10i2.9127

Nielsen, T. (2018). The intrinsic and extrinsic motivation subscales of the motivated strategies for learning questionnaire: A Rasch-based construct validity study. Cogent Education, 5(1), 1504485. https://doi.org/10.1080/2331186X.2018.1504485

Nopiah, Z. M., Rosli, S., Baharin, M. N., Othman, H., & Ismail, A. (2012). Evaluation of pre-assessment method on improving student’s performance in complex analysis course. Asian Social Science, 8(16), 134–139. https://doi.org/10.5539/ass.v8n16p134

Pesudovs, K., Burr, J. M., Harley, C., & Elliott, D. B. (2007). The development, assessment, and selection of questionnaires. Optometry and Vision Science, 84(8), 663–674. https://doi.org/10.1097/OPX.0b013e318141fe75

Pesudovs, K., Garamendi, E., Keeves, J. P., & Elliott, D. B. (2003). The activities of daily vision scale for cataract surgery outcomes: Re-evaluating validity with Rasch analysis. Investigative Ophthalmology and Visual Science, 44(7), 2892–2899. https://doi.org/10.1167/iovs.02-1075

Planinic, M., Boone, W. J., Susac, A., & Ivanjek, L. (2019). Rasch analysis in physics education research: Why measurement matters. Physical Review Physics Education Research, 15(2), 020111. https://doi.org/10.1103/PhysRevPhysEducRes.15.020111

Plucker, J. A., Qian, M., & Schmalensee, S. L. (2014). Is what you see what you really get? Comparison of scoring techniques in the assessment of real-world divergent thinking. Creativity Research Journal, 26(2), 135–143. https://doi.org/10.1080/10400419.2014.901023

Pontoppidan, M., Nielsen, T., & Kristensen, I. H. (2018). Psychometric properties of the Danish parental stress scale: Rasch analysis in a sample of mothers with infants. PLoS ONE, 13(11), e0205662. https://doi.org/10.1371/journal.pone.0205662

Rifbjerg-Madsen, S., Wæhrens, E. E., Danneskiold-Samsøe, B., & Amris, K. (2017). Psychometric properties of the painDETECT questionnaire in rheumatoid arthritis, psoriatic arthritis and spondyloarthritis: Rasch analysis and test-retest reliability. Health and Quality of Life Outcomes, 15(1), 110. https://doi.org/10.1186/s12955-017-0681-1

Riyanti, A., Widiyatmoko, A., & Wusqo, I. U. (2016). pengaruh model pembelajaran kooperatif tipe Team Assisted Individualization berbantuan peta konsep terhadap hasil belajar dan keterampilan berpikir kritis siswa SMP tema Kalor. Unnes Science Education Journal, 5(2), 70805795–70850229. http://journal.unnes.ac.id/sju/index.php/usej

Runco, M. A., & Acar, S. (2012). Divergent thinking as an indicator of creative potential. Creativity Research Journal, 24(1), 66–75. https://doi.org/10.1080/10400419.2012.652929

Runco, M. A., & Albert, R. S. (1985). The reliability and validity of ideational originality in the divergent thinking of academically gifted and nongifted children. Educational and Psychological Measurement, 45(3), 483–501. https://doi.org/10.1177/001316448504500306

Subroto, G., Agust, S., Angela, A., Dezar, A., Zahra, D., Mirarizka, D., Rianto, F., Rayani, V., & Candra, M. (2022). Coastal students’ perspectives on digital reading comprehension: A Rasch model analysis. In Proceedings of the 1st International Conference on Maritime Education, ICOME 2021, 3-5 November 2021, Tanjungpinang, Riau Islands, Indonesia. https://doi.org/10.4108/eai.3-11-2021.2314832

Sulastri, A., Badruzsaufari, B., Dharmono, D., Aufa, M. N., & Saputra, M. A. (2022). Development of Science handouts based on critical thinking skills on the topic of the Human Digestive System. Jurnal Penelitian Pendidikan IPA, 8(2), 475–480. https://doi.org/10.29303/jppipa.v8i2.1156

Sumintono, B. (2018). Rasch model measurements as tools in assessment for learning. In Proceedings of the 1st International Conference on Education Innovation (ICEI 2017), pp. 38-42. https://doi.org/10.2991/icei-17.2018.11

Sumintono, B., & Widhiarso, W. (2015). Aplikasi pemodelan Rasch pada assessment pendidikan. Trim Komunikata.

Susongko, P., Yuenyong, C., & Zainudin, A. (2022). Buddhist critical thinking assessment using Rasch model. Kasetsart Journal of Social Sciences, 43(2), 285–292. https://doi.org/10.34044/j.kjss.2022.43.2.04

Vincent, J. I., MacDermid, J. C., King, G. J. W., & Grewal, R. (2015). Rasch analysis of the Patient Rated Elbow Evaluation questionnaire. Health and Quality of Life Outcomes, 13(1), 84. https://doi.org/10.1186/s12955-015-0275-8

Wahyudiati, D. (2022). Critical thinking skills and scientific attitudes of pre-service Chemistry teachers through the implementation of problem-based learning model. Jurnal Penelitian Pendidikan IPA, 8(1), 216–221. https://doi.org/10.29303/jppipa.v8i1.1278

Widoyoko, E. P. (2009). Evaluasi program pembelajaran. Pustaka Pelajar.

Widyaningsih, W., & Yusuf, I. (2018). Project based learning model based on simple teaching tools and critical thinking skills. Physics Education Journal, 1(1), 12–21. https://doi.org/10.37891/kpej.v1i1.33

Zwick, R., Thayer, D. T., & Lewis, C. (1999). An empirical Bayes approach to Mantel-Haenszel DIF analysis. Journal of Educational Measurement, 36(1), 1-28. https://www.jstor.org/stable/1435320

DOI: https://doi.org/10.21831/pep.v27i1.58873