Technology-enhanced learning for statistical graph interpretation: An item response theory analysis of learning outcomes

Authors

  • Bambang Subali Universitas Negeri Semarang, Indonesia https://orcid.org/0000-0002-7427-4245
  • Ridho Adi Negoro Universitas Negeri Semarang, Indonesia
  • Ellianawati Universitas Negeri Semarang, Indonesia https://orcid.org/0000-0003-3736-8979
  • Pratiwi Dwijananti Universitas Negeri Semarang, Indonesia
  • Aulia Silvina Anandita Universitas Negeri Semarang, Indonesia
  • Natalia Erna Setyaningsih Universitas Negeri Semarang, Indonesia
  • Siswanto Universitas Tidar, Indonesia

DOI:

https://doi.org/10.21831/reid.v11i2.89666

Keywords:

technology-enhanced learning, statistical graph interpretation, item response theory, instrument validation

Abstract

This study aimed to examine the effectiveness of technology-enhanced learning (TEL) in improving students’ statistical graph interpretation skills through a rigorous Item Response Theory (IRT) analysis. Employing a quasi-experimental pretest–posttest control group design, the research involved 120 undergraduate students from four classes, equally divided into experimental and control groups. The experimental groups received TEL-based instruction featuring interactive graph visualizations and automated feedback, while the control groups followed conventional lectures and exercises over seven sessions. Data were collected using a 60-item multiple-choice test covering bar charts, histograms, boxplots, and scatterplots, which was content-validated by experts and trialed for clarity, yielding high reliability (Cronbach’s α = 0.833). Construct validity was ensured through unidimensionality and invariance testing, confirmed by eigenvalue and DETECT analysis. Data analysis applied IRT to calibrate item parameters discrimination (a), difficulty (b), and guessing (c) and to estimate students’ latent abilities (θ). Model comparison identified the 3PL model as the best fit, capturing both difficulty variation and guessing behavior. Calibration results showed that most items exhibited satisfactory psychometric quality, supporting the robustness of the instrument. Findings revealed that TEL groups achieved a nearly one-logit gain in ability from pretest to posttest, significantly higher than the minimal improvement observed in the control groups, as confirmed by independent t-tests and normalized gain analysis. These results indicate that TEL substantially strengthens students’ ability to interpret statistical graphs while demonstrating the diagnostic value of IRT in evaluating both item quality and learning effectiveness.

References

Al-Ansi, A. M., Jaboob, M., Garad, A., & Al-Ansi, A. (2023). Analyzing Augmented Reality (AR) and Virtual Rreality (VR) Recent Development in Education. Social Sciences and Humanities Open, 8(1), 1–10. https://doi.org/10.1016/j.ssaho.2023.100532

Altindis, N., Bowe, K. A., Couch, B., Bauer, C. F., & Aikens, M. L. (2024). Exploring the Role of Disciplinary Knowledge in Students’ Covariational Reasoning During Ggraphical Interpretation. International Journal of STEM Education, 11(1). https://doi.org/10.1186/s40594-024-00492-5

Binali, T., Chang, C. H., Chang, Y. J., & Chang, H. Y. (2024). High School and College Students’ Graph-Interpretation Competence in Scientific and Daily Contexts of Data Visualization. Science and Education, 33(3), 763–785. https://doi.org/10.1007/s11191-022-00406-3

Cantó-Cerdán, M., Cacho-Martínez, P., Lara-Lacárcel, F., & García-Muñoz, Á. (2021). Rasch analysis for development and reduction of Symptom Questionnaire for Visual Dysfunctions (SQVD). Scientific Reports, 11(1). https://doi.org/10.1038/s41598-021-94166-9

Chang, H. Y., Chang, Y. J., & Tsai, M. J. (2024). Strategies and Difficulties During Students’ Construction of Data Visualizations. International Journal of STEM Education, 11(1), 1–22. https://doi.org/10.1186/s40594-024-00463-w

Cooper, L. L., & Shore, F. S. (2008). Students’ Misconceptions in Interpreting Center and Variability of Data Represented Via Histograms and Stem-and-leaf Plots. Journal of Statistics Education, 16(2), 1–13. https://doi.org/10.1080/10691898.2008.11889559

Creswell, J. W., & Guetterman, T. C. (2024). Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research (7th ed.). Pearson.

Finch, W. H., & Jeffers, H. (2016). A Q3-Based Permutation Test for Assessing Local Independence. Applied Psychological Measurement, 40(2), 157–160. https://doi.org/10.1177/0146621615622635

Halliday, S. D., Makler, C., McKee, D., & Papadopoulou, A. (2024). Improving Student Comprehension Through Interactive Model Visualization. International Review of Economics Education, 47, 1–6. https://doi.org/10.1016/j.iree.2024.100296

Han, Y., Jiang, Z., Ouyang, J., Xu, L., & Cai, T. (2022). Psychometric Evaluation of a National Exam for Clinical Undergraduates. Frontiers in Medicine, 9(1), 1–10. https://doi.org/10.3389/fmed.2022.1037897

Himelfarb, I. (2019). A Primer on Standardized Testing: History, Measurement, Classical Test Theory, Item Response Theory, and Equating. Journal of Chiropractic Education, 33(2), 151–163. https://doi.org/10.7899/JCE-18-22

İlkörücü, Ş., & Broutin, M. S. T. (2022). Evaluation of Pre-Service Science Teachers’ Visual and Cognitive Constructions of A Line Graph. Uludağ Üniversitesi Eğitim Fakültesi Dergisi, 35(3), 647–668. https://doi.org/10.19171/uefad.1099612

Joo, S., Ali, U., Robin, F., & Shin, H. J. (2022). Impact of Differential Item Functioning on Group Score Reporting in The Context of Large-Scale Assessments. Large-Scale Assessments in Education, 10(1), 1–21. https://doi.org/10.1186/s40536-022-00135-7

Jungjohann, J., Gebhardt, M., & Scheer, D. (2022). Understanding and improving teachers’ graph literacy for data-based decision-making via video intervention. Frontiers in Education, 7(1), 1–18. https://doi.org/10.3389/feduc.2022.919152

Krishnan, S., & Idris, N. (2018). Using Partial Credit Model to Improve the Quality of an Instrument. International Journal of Evaluation and Research in Education (IJERE), 7(4), 313. https://doi.org/10.11591/ijere.v7i4.15146

Lee, S., Choi, Y.-J., & Kim, H.-S. (2021). The Accurate Measurement of Students’ Learning in E-Learning Environments. Applied Sciences, 11(1), 1–11. https://doi.org/10.3390/app11219946

Luo, J., Zheng, C., Yin, J., & Teo, H. H. (2025). Design and Assessment of AI-based Learning Tools in Higher Education: a Systematic Review. International Journal of Educational Technology in Higher Education, 22(1), 1–27. https://doi.org/10.1186/s41239-025-00540-2

Mallinson, T., Kozlowski, A. J., Johnston, M. V., Weaver, J., Terhorst, L., Grampurohit, N., Juengst, S., Ehrlich-Jones, L., Heinemann, A. W., Melvin, J., Sood, P., & Van de Winckel, A. (2022). Rasch Reporting Guideline for Rehabilitation Research (RULER): the RULER Statement. Archives of Physical Medicine and Rehabilitation, 103(7), 1477–1486. https://doi.org/10.1016/j.apmr.2022.03.013

Nwagwu, W. E. (2024). Mapping the field of global research on data literacy: Key and emerging issues and the library connection. International Federation of Library Associations and Institutions, 50(3), 491–510. https://doi.org/10.1177/03400352241257669

Ongena, G. (2023). Data literacy for improving governmental performance: A competence-based approach and multidimensional operationalization. Digital Business, 3(1), 1–12. https://doi.org/10.1016/j.digbus.2022.100050

Rufiana, I. S., Arifin, S., Randy, M. Y., & Amaliya, F. N. (2024). Analysis of Student Errors in Solving Numeracy Literacy Problems of Graph Representation Model in Elementary School. Al Ibtida: Jurnal Pendidikan Guru MI, 11(2), 300–319. https://doi.org/10.24235/al.ibtida.snj.v11i2.18720

Sailer, M., Maier, R., Berger, S., Kastorff, T., & Stegmann, K. (2024). Learning activities in technology-enhanced learning: A systematic review of meta-analyses and second-order meta-analysis in higher education. Learning and Individual Differences, 112. https://doi.org/10.1016/j.lindif.2024.102446

Sethar, W. A., Pitafi, A., Bhutto, A., Nassani, A. A., Haffar, M., & Kamran, S. M. (2022). Application of Item Response Theory (IRT)-Graded Response Model (GRM) to Entrepreneurial Ecosystem Scale. Sustainability (Switzerland), 14(9). https://doi.org/10.3390/su14095532

Sviridova, E., Yastrebova, E., Bakirova, G., & Rebrina, F. (2023). Immersive Technologies as an Innovative Tool to Increase Academic Success and Motivation in Higher Education. Frontiers in Education, 8(1), 1–10. https://doi.org/10.3389/feduc.2023.1192760

Tan, A. J. Y., Davies, J., Nicolson, R. I., & Karaminis, T. (2023). A Technology-enhanced Learning Intervention for Statistics in Higher Education Using Bite-sized Video-Based Learning and Precision Teaching. Research and Practice in Technology Enhanced Learning, 18(1), 1–7. https://doi.org/10.58459/rptel.2023.18001

Verdú-Soriano, J., & González-de la Torre, H. (2024). Rasch Analysis Implementation in Nursing Research: A Methodological Approach. Enfermeria Clinica, 34(6), 493–506. https://doi.org/10.1016/j.enfcli.2024.10.005

Vermunt, J. D. (2023). Understanding, Measuring and Improving Simulation-based Learning in Higher Education: Student and Teacher Learning Perspectives. Learning and Instruction, 86(1), 1–4. https://doi.org/10.1016/j.learninstruc.2023.101773

Wardani, R. T., Nuraeni, E., & Diana, S. (2025). Rasch Model Analysis of Essay Questions to Measure Literacy and Numeracy Skills in Plant and Animal Bioprocess Topics based on AKM. REID (Research and Evaluation in Education), 11(1), 29–44. https://doi.org/10.21831/reid.v11i1.85614

Wenglein, J. S., Heidenreich, A., & Friederichs, H. (2025). Assessment of Graph Literacy Among German Medical Students– an Observational Cross-sectional Survey Study. BMC Medical Education, 25(1), 1–10. https://doi.org/10.1186/s12909-025-07206-7

Wind, S. A. (2023). Detecting Rating Scale Malfunctioning With the Partial Credit Model and Generalized Partial Credit Model. Educational and Psychological Measurement, 83(5), 953–983. https://doi.org/10.1177/00131644221116292

Zhang, X., Qian, W., & Chen, C. (2024). The Effect of Digital Technology Usage on Higher Vocational Student Satisfaction: The Mediating Role of Learning Experience and Learning Engagement. Frontiers in Education, 9(1), 1–13. https://doi.org/10.3389/feduc.2024.1508119

Published

2025-12-05

How to Cite

Subali, B., Negoro, R. A., Ellianawati, Dwijananti, P., Anandita, A. S., Setyaningsih, N. E., & Siswanto. (2025). Technology-enhanced learning for statistical graph interpretation: An item response theory analysis of learning outcomes. REID (Research and Evaluation in Education), 11(2). https://doi.org/10.21831/reid.v11i2.89666

Issue

Vol. 11 No. 2 (2025)

Section

Articles

Citation Check

License

Copyright (c) 2025 REID (Research and Evaluation in Education)

Creative Commons License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

The authors submitting a manuscript to this journal agree that, if accepted for publication, copyright publishing of the submission shall be assigned to REID (Research and Evaluation in Education). However, even though the journal asks for a copyright transfer, the authors retain (or are granted back) significant scholarly rights.

The copyright transfer agreement form can be downloaded here: [REID Copyright Transfer Agreement Form] 


The copyright form should be signed originally and sent to the Editorial Office through email to reid.ppsuny@uny.ac.id

 

Creative Commons License
REID (Research and Evaluation in Education) by http://journal.uny.ac.id/index.php/reid is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.