Psychometric evaluation of diagnostic test instruments on physics wave material

Muh. Asriadi AM; Anna Isabela Sanam

doi:10.21831/pep.v29i1.84655

Psychometric evaluation of diagnostic test instruments on physics wave material

Muh. Asriadi AM, Universitas Pendidikan Indonesia, Indonesia
Anna Isabela Sanam, Institute of Business, Timor-Leste

10.21831/pep.v29i1.84655

Abstract

This study aims to evaluate the psychometric quality of diagnostic test instruments on wave material in physics learning, with a focus on their validity and reliability. The instrument was validated by seven experts using the content validation method, which resulted in an average validity score of 0.849, indicating that the instrument is very valid. Confirmatory factor analysis (CFA) was conducted to test the model fit and validity of the instrument structure. The KMO test results showed a figure of 0.946, indicating the suitability of the data for factor analysis, while the Bartlett Test of Sphericity results confirmed that the intercorrelation data was not an identity matrix. The tested model showed a good fit to the data, with a CFI value of 0.948, TLI of 0.935, and RMSEA of 0.072. The model parameter estimates showed a significant relationship between the latent variables and the observed items, with a strong influence from the "Wave" variable. Although the Chi-square test results indicated a statistically poor fit of the model, this result could be influenced by the large sample size. Limitations of this study include the limited sample size of seven validators, which may not cover all perspectives. This study suggests that further research involving a larger number of validators and more in-depth analysis should be conducted to strengthen the results obtained. The results of this study are expected to contribute to the development of more effective diagnostic test instruments for physics teaching at the secondary school level.

Keywords

Aiken's V; EFA; CFA; Reliability; diagnostic test

References

Abdillah, M. B., Rahardjo, T. J., Martono, & Prihatin, T. (2021). The influence of transformational leadership, organizational culture, and rewards through an commitment to integrated goal to the performance of private higher education lecturers in the central java region. Proceeding ISET (2021) Universitas Negeri Semarang International Conference on Science, Education and Technology, 7(1), 812–816. http://ezproxy.umgc.edu/login?url=https://search.ebscohost.com/login.aspx?direct=true&db=edb&AN=160601116&site=eds-live&scope=site

AM, M. A., & Istiyono, E. (2022). Multiple representation ability of high school students in physics: A study of modern response theory. Thabiea: Journal of Natural Science Teaching, 5(1), 85–97. https://doi.org/10.21043/thabiea.v5i1.12550

Asriadi, M., & Hadi, S. (2021). Analysis of the quality of the formative test items for physics learning using the rasch model in the 21st century learning. JIPF (Jurnal Ilmu Pendidikan Fisika), 6(2), 158–166. https://doi.org/10.26737/jipf.v6i2.2030

Blessing, A., Emmanuel, N., & Chinyere, E. (2021). Effect of differentiated instruction on students’ achievement in geometry. International Journal for Research in Applied Sciences and Biotechnology, 8(3), 6–12. https://doi.org/https://doi.org/10.31033/ijrasb.8.3.2

Burak, D., & Gültekin, M. (2021). Verbal-visual learning styles scale: Developing a scale for primary school students. International Journal on Social and Education Sciences, 3(2), 287–303. https://doi.org/10.46328/ijonses.171

Burkholder, E., Wang, K., & Wieman, C. (2021). Validated diagnostic test for introductory physics course placement. Physical Review Physics Education Research, 17(1), 10127. https://doi.org/10.1103/PhysRevPhysEducRes.17.010127

Creswell, J. W. (2022). Research design: Qualitative, quantitative, and mixed methods approaches. SAGE Publications.

Hadi, S., Haryanto, H., AM, M. A., Marlina, M., & Rahim, A. (2022). Developing classroom assessment tool using learning management system-based computerized adaptive test in vocational high schools. Journal of Education Research and Evaluation, 6(1), 143–155. https://doi.org/10.23887/jere.v6i1.35630

Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2017). Multivariate data analysis. In Pearson (Seventh). https://doi.org/10.1002/9781118895238.ch8

Hair, J. F. J., Black, W. C., Babin, B. J., & Anderson, R. E. (2019). Multivariate data analysis (Eighth). Annabel Ainscow.

Haryanto, H., Kholis, N., Hadi, S., Apino, E., Asriadi AM, M., & Trizkyana, C. K. (2023). Determinant factors affecting the research performance of lecturers receiving external funds. Research and Evaluation in Education, 9(2), 198–209. https://doi.org/10.21831/reid.v9i2.68457

Helmi, S., Sofyan, Y., Chantika, T., & Muh Asriadi, A. M. (2025). Linking Culture to Performance: How Organizational Culture Drives Employee Success at PT Cyberindo Aditama (CBN). International Research Journal of Multidisciplinary Scope, 6(1), 441–450. https://doi.org/10.47857/irjms.2025.v06i01.02304

Homjan, S., Sri-ngan, K., & Homjan, W. (2022). Construction of diagnostic test in mathematics on addition and subtraction basic for primary students. Journal of Education and Learning, 11(3), 88–94. https://doi.org/10.5539/jel.v11n3p88

Hyland, D., & O’Shea, A. (2022). The nature and prevalence of diagnostic testing in mathematics at tertiary-level in Ireland. Teaching Mathematics and Its Applications, 41(1), 32–50. https://doi.org/10.1093/teamat/hrab006

Istiyono, E. (2022). Diagnostic tests as an important pillar in today’s physics learning: Four-tier diagnostic test a comprehensive diagnostic test solution. Journal of Physics: Conference Series, 2392(1), 0–8. https://doi.org/10.1088/1742-6596/2392/1/012001

Istiyono, E., Sunu, W., Fenditasari, K., Rai, M., & Saepuzaman, D. (2023). The development of a four-tier diagnostic test based on modern test theory in physics education. European Journal of Educational Research, 12(1), 371–385. https://doi.org/10.12973/eu-jer.12.1.371

Johnson, R. B., & Christensen, L. (2017). Educational research: Quantitative, qualitative, and mixed approaches. In SAGE Publications, Inc.

Kaniawati, I., Fratiwi, N. J., Danawan, A., Suyana, I., Samsudin, A., & Suhendi, E. (2019). Analyzing students’ misconceptions about newton’s laws through four-tier newtonian test (FTNT). Journal of Turkish Science Education, 16(1), 110–122. https://doi.org/10.12973/tused.10269a

Kastriti, E., Kalogiannakis, M., Psycharis, S., & Vavougios, D. (2022). The teaching of Natural Sciences in kindergarten based on the principles of STEM and STEAM approach. Advances in Mobile Learning Educational Research, 2(1), 268–277. https://doi.org/10.25082/amler.2022.01.011

Martiana, A., Istiyono, E., & Widihastuti, W. (2022). Critical sociology in the development of HOTS-oriented cognitive assessment instruments. Journal of Social Studies (JSS), 18(2), 197–206. https://doi.org/10.21831/jss.v18i2.51430

Navarro, D., Foxcroft, D., & Faulkenberry, T. J. (2019). Learning stafisticks with JASP: A tutorial for psychology students and other beginners. (Version 1/2).

Rahman, K. A., Hasan, M. K., Namaziandost, E., & Ibna Seraj, P. M. (2021). Implementing a formative assessment model at the secondary schools: attitudes and challenges. Language Testing in Asia, 11(1), 1–18. https://doi.org/10.1186/s40468-021-00136-3

Scoulas, J. M., Aksu Dunya, B., & De Groote, S. L. (2021). Validating students’ library experience survey using rasch model. Library and Information Science Research, 43(1), 101071. https://doi.org/10.1016/j.lisr.2021.101071

Shanmugam, S. K. S. (2018). Determining gender differential item functioning for mathematics in coeducational school culture. Malaysian Journal of Learning and Instruction, 15(2), 83–109. https://doi.org/10.32890/mjli2018.15.2.4

Spiller, L., & Tuten, T. (2019). Assessing the pedagogical value of branded digital marketing certification programs. Journal of Marketing Education, 41(2), 77–90. https://doi.org/10.1177/0273475318822686

Tabatabaee-Yazdi, M., Motallebzadeh, K., Ashraf, H., & Baghaei, P. (2018). Development and validation of a teacher success questionnaire using the rasch model. International Journal of Instruction, 11(2), 129–144. https://doi.org/10.12973/iji.2018.11210a

Tomczyk, Ł., & Eger, L. (2020). Online safety as a new component of digital literacy for young people. Integration of Education, 24(2), 172–184. https://doi.org/10.15507/1991-9468.099.024.202002.172-184

Watkins, M. W. (2018). Exploratory factor analysis: A guide to best practice. Journal of Black Psychology, 44(3), 219–246. https://doi.org/10.1177/0095798418771807

DOI: https://doi.org/10.21831/pep.v29i1.84655