Psychometric Validating a Video-Based Rubric for Physics Teacher Assessment Using the Rasch Model
DOI:
https://doi.org/10.21831/jipi.v11i2.82060Keywords:
physics prospective teachers, professional competence, Rasch model, video-based assessmentAbstract
This study aims to validate the psychometric properties of a video-based assessment rubric designed to evaluate the pedagogical and professional competencies of physics prospective teachers. The validation process employed the Rasch model to examine item reliability, person reliability, item fit, category functioning, and the separation index. Data were collected from 61 participants using a 25-item rubric that covered various aspects, including Pancasila values, instructional structure, student-centered learning, professional ethics, and evaluation of learning. The results showed that the item reliability was high (0.83), while person reliability was moderate (0.75), with a person separation index of 1.75. Some items exhibited misfit with the Rasch model, indicating the need for revision or refinement. Analysis of category functioning revealed overlapping probabilities between adjacent categories, suggesting that the response scale could benefit from more precise distinctions. Overall, the rubric demonstrated acceptable psychometric quality for assessing the competencies of physics prospective teachers, although several items and scale structures require improvement to enhance measurement precision
References
Adhantoro, M. S., Gunawan, D., Prayitno, H. J., Riyanti, R. F., & Jufriansah, A. (2024). Strategies to Enhance Literacy and Access to Muhammadiyah Information through ChatMu Innovation. International Journal of Religion, 5(11), 2503–2520. https://doi.org/10.61707/7hqaep83
Adipat, S. (2021). Developing Technological Pedagogical Content Knowledge (TPACK) through Technology-Enhanced Content and Language-Integrated Learning (T-CLIL) Instruction. Education and Information Technologies, 26(5), 6461–6477. https://doi.org/10.1007/s10639-021-10648-3
Agricola, B. T., van der Schaaf, M. F., Prins, F. J., & van Tartwijk, J. (2022). The development of research supervisors’ pedagogical content knowledge in a lesson study project. Educational Action Research, 30(2), 261–280. https://doi.org/10.1080/09650792.2020.1832551
Akyuz, D. (2018). Measuring technological pedagogical content knowledge (TPACK) through performance assessment. Computers & Education, 125, 212–225. https://doi.org/10.1016/j.compedu.2018.06.012
Aryan, Hegade, P., & Shettar, A. (2023). Effectiveness of Computational Thinking in Problem Based Learning. Journal of Engineering Education Transformations, 36(S2), 179–185. https://doi.org/10.16920/jeet/2023/v36is2/23025
Bai, H., Wang, X., & Zhao, L. (2021). Effects of the Problem-Oriented Learning Model on Middle School Students’ Computational Thinking Skills in a Python Course. Frontiers in Psychology, 12. https://doi.org/10.3389/fpsyg.2021.771221
Bayram‐Jacobs, D., Henze, I., Evagorou, M., Schwartz, Y., Aschim, E. L., Alcaraz‐Dominguez, S., Barajas, M., & Dagan, E. (2019). Science teachers’ pedagogical content knowledge development during enactment of socioscientific curriculum materials. Journal of Research in Science Teaching, 56(9), 1207–1233. https://doi.org/10.1002/tea.21550
Cetin-Dindar, A., Boz, Y., Yildiran Sonmez, D., & Demirci Celep, N. (2018). Development of pre-service chemistry teachers’ technological pedagogical content knowledge. Chemistry Education Research and Practice, 19(1), 167–183. https://doi.org/10.1039/C7RP00175D
Cheah, Y. H., Chai, C. S., & Toh, Y. (2019). Traversing the context of professional learning communities: development and implementation of Technological Pedagogical Content Knowledge of a primary science teacher. Research in Science & Technological Education, 37(2), 147–167. https://doi.org/10.1080/02635143.2018.1504765
Clausen, S. W. (2018). Exploring the pedagogical content knowledge of Danish geography teachers: teaching weather formation and climate change. International Research in Geographical and Environmental Education, 27(3), 267–280. https://doi.org/10.1080/10382046.2017.1349376
Delgado-Rebolledo, R., & Zakaryan, D. (2020). Relationships Between the Knowledge of Practices in Mathematics and the Pedagogical Content Knowledge of a Mathematics Lecturer. International Journal of Science and Mathematics Education, 18(3), 567–587. https://doi.org/10.1007/s10763-019-09977-0
Diamah, A., Rahmawati, Y., Paristiowati, M., Fitriani, E., Irwanto, I., Dobson, S., & Sevilla, D. (2022). Evaluating the effectiveness of technological pedagogical content knowledge-based training program in enhancing pre-service teachers’ perceptions of technological pedagogical content knowledge. Frontiers in Education, 7. https://doi.org/10.3389/feduc.2022.897447
DURAN, M., USAK, M., HSIEH, M.-Y., & UYGUN, H. (2021). A New Perspective on Pedagogical Content Knowledge: Intellectual and Emotional Characteristics of Science Teachers. Journal of Research and Social Intervention, 72, 9–32. https://doi.org/10.33788/rcis.72.1
Eckes, T. (2019). Many-facet Rasch measurement: Implications for rater-mediated language assessment. In Quantitative Data Analysis for Language Assessment Volume I (pp. 153–175). Routledge. https://doi.org/10.4324/9781315187815-8
Eckes, T. (2019). Many-facet Rasch measurement: Implications for rater-mediated language assessment. In Quantitative data analysis for language assessment Volume I (pp. 153–175). Routledge. https://doi.org/10.4324/9781315187815-8
Ettekal, I., & Shi, Q. (2020). Developmental trajectories of teacher-student relationships and longitudinal associations with children’s conduct problems from Grades 1 to 12. Journal of School Psychology, 82, 17–35. https://doi.org/10.1016/j.jsp.2020.07.004
Fitriansyah, R., Fatinah, L., & Syahril, M. (2020). Critical Review: Professional Development Programs to Face Open Educational Resources in Indonesia. Indonesian Journal on Learning and Advanced Education (IJOLAE), 2(2), 109–119. https://doi.org/10.23917/ijolae.v2i2.9662
Hsu, L., & Chen, Y.-J. (2019). Examining teachers’ technological pedagogical and content knowledge in the era of cloud pedagogy. South African Journal of Education, 39(S2), 1–13. https://doi.org/10.15700/saje.v39ns2a1572
Hubbard, A. (2018). Pedagogical content knowledge in computing education: a review of the research literature. Computer Science Education, 28(2), 117–135. https://doi.org/10.1080/08993408.2018.1509580
Hursen, C. (2021). The Effect of Problem-Based Learning Method Supported by Web 2.0 Tools on Academic Achievement and Critical Thinking Skills in Teacher Education. Technology, Knowledge and Learning, 26(3), 515–533. https://doi.org/10.1007/s10758-020-09458-2
James, F., & Augustin, D. S. (2018). Improving teachers’ pedagogical and instructional practice through action research: potential and problems. Educational Action Research, 26(2), 333–348. https://doi.org/10.1080/09650792.2017.1332655
Karousiou, C., Hajisoteriou, C., & Angelides, P. (2019). Teachers’ professional identity in super-diverse school settings: teachers as agents of intercultural education. Teachers and Teaching, 25(2), 240–258. https://doi.org/10.1080/13540602.2018.1544121
Kholili, M. I., Dewantoro, A., Surur, N., & Hapsari, N. T. (2024). The 21st-century skills scales: many facet Rasch measurements. International Journal of Evaluation and Research in Education (IJERE), 13(3), 1424. https://doi.org/10.11591/ijere.v13i3.26651
Kholili, M. I., Dewantoro, A., Surur, N., & Hapsari, N. T. (2024). The 21st-century skills scales: Many facet Rasch measurements. International Journal of Evaluation and Research in Education, 13(3), 1424. https://doi.org/10.11591/ijere.v13i3.26651
König, J., Doll, J., Buchholtz, N., Förster, S., Kaspar, K., Rühl, A.-M., Strauß, S., Bremerich-Vos, A., Flade, I., & Kaiser, G. (2018). Pedagogical knowledge versus didactic knowledge? Journal of Educational Science, 21(3), 1–38. https://doi.org/10.1007/s11618-017-0765-z
Larson, K. E., Hirsch, S. E., McGraw, J. P., & Bradshaw, C. P. (2020). Preparing Preservice Teachers to Manage Behavior Problems in the Classroom: The Feasibility and Acceptability of Using a Mixed-Reality Simulator. Journal of Special Education Technology, 35(2), 63–75. https://doi.org/10.1177/0162643419836415
LEE, J., KIM, J. B., & KIM*, J. B. (2018). Effects of the Experience in Developing Physics Teaching Materials Based on Computational Thinking for Improvement of Science Teachers’ and Pre-service Teachers’ Technological Pedagogical and Content Knowledge (TPACK). New Physics: Sae Mulli, 68(2), 202–216. https://doi.org/10.3938/NPSM.68.202
Li, B., Zhao, Y., & Zhang, H. (2018). Video Analysis of the Influence of Intelligent Media Application on Teachers’ Knowledge Structure: A Case Study of Physics Lesson at Middle School. 2018 International Joint Conference on Information, Media and Engineering (ICIME), 249–254. https://doi.org/10.1109/ICIME.2018.00059
Li, J., Shi, Z., & Xue, E. (2020). The problems, needs and strategies of rural teacher development at deep poverty areas in China: Rural schooling stakeholder perspectives. International Journal of Educational Research, 99, 101496. https://doi.org/10.1016/j.ijer.2019.101496
Nurhasanah, M., Suprapto, P. K., & Ardiansyah, R. (2024). The effectiveness of problem-based learning assisted by Articulate Storyline interactive students’ critical thinking skills. Jurnal Inovasi Pendidikan IPA, 10(1), 1–12. https://doi.org/10.21831/jipi.v10i1.64847
Mayer, P., & Girwidz, R. (2019). Physics Teachers’ Acceptance of Multimedia Applications—Adaptation of the Technology Acceptance Model to Investigate the Influence of TPACK on Physics Teachers’ Acceptance Behavior of Multimedia Applications. Frontiers in Education, 4. https://doi.org/10.3389/feduc.2019.00073
Melo, L., Cañada-Cañada, F., González-Gómez, D., & Jeong, J. S. (2020). Exploring Pedagogical Content Knowledge (PCK) of Physics Teachers in a Colombian Secondary School. Education Sciences, 10(12), 362. https://doi.org/10.3390/educsci10120362
Metz, M. (2021). Pedagogical Content Knowledge for Teaching Critical Language Awareness: The Importance of Valuing Student Knowledge. Urban Education, 56(9), 1456–1484. https://doi.org/10.1177/0042085918756714
Nurulsari, N., Abdurrahman, Maulina, H., Sukamto, I., & Umam, R. (2020). Exploring the Prospective of Pre-Service Physics Teacher’s Pedagogical Content Knowledge: A Case Study. Journal of Physics: Conference Series, 1467(1), 012023. https://doi.org/10.1088/1742-6596/1467/1/012023
Nurulsari, N., Abdurrahman, Maulina, H., Sukamto, I., & Umam, R. (2020). Exploring the prospective of pre-service physics teacher’s pedagogical content knowledge: A case study. Journal of Physics: Conference Series, 1467(1), 012023. https://doi.org/10.1088/1742-6596/1467/1/012023
Putra, P. D. A., & Narulita, E. (2023). Teacher professional knowledge: The implementation of STEM pedagogical content knowledge in pandemic era. 060014. https://doi.org/10.1063/5.0111357
Rieu, A., Leuders, T., & Loibl, K. (2022). Teachers’ diagnostic judgments on tasks as information processing – The role of pedagogical content knowledge for task diagnosis. Teaching and Teacher Education, 111, 103621. https://doi.org/10.1016/j.tate.2021.103621
SA, N. H., Suyanto, S., Arifi, A., Putranta, H., & Azizah, A. N. M. (2021). Experiences of Participants in Teacher Professional Education on Obtaining Soft Skills: A Case Study in Indonesia. European Journal of Educational Research, volume-10-2021(volume-10-issue-1-january-2021), 313–325. https://doi.org/10.12973/eu-jer.10.1.313
Scherer, R., Tondeur, J., Siddiq, F., & Baran, E. (2018). The importance of attitudes toward technology for pre-service teachers’ technological, pedagogical, and content knowledge: Comparing structural equation modeling approaches. Computers in Human Behavior, 80, 67–80. https://doi.org/10.1016/j.chb.2017.11.003
Sorge, S., Kröger, J., Petersen, S., & Neumann, K. (2019). Structure and development of physics prospective teachers’ professional knowledge. International Journal of Science Education, 41(7), 862–889. https://doi.org/10.1080/09500693.2017.1346326
Splett, J. W., Garzona, M., Gibson, N., Wojtalewicz, D., Raborn, A., & Reinke, W. M. (2019). Teacher Recognition, Concern, and Referral of Children’s Internalizing and Externalizing Behavior Problems. School Mental Health, 11(2), 228–239. https://doi.org/10.1007/s12310-018-09303-z
Styck, K. M., Anthony, C. J., Flavin, A., Riddle, D., & LaBelle, B. (2021). Are ratings in the eye of the beholder? A non-technical primer on many facet Rasch measurement to evaluate rater effects on teacher behavior rating scales. Journal of School Psychology, 86, 198–221. https://doi.org/10.1016/j.jsp.2021.01.001
Sudirman, S. (2019). The 21st-Century Teacher: Teacher’s Competence Within the Character Education Framework Towards A Cultural-Oriented Development and Promoting Tolerance. International Education Studies, 12(8), 21–25. https://doi.org/10.5539/ies.v12n8p21
Syahmani, S., Hafizah, E., Sauqina, S., Adnan, M. Bin, & Ibrahim, M. H. (2021). STEAM Approach to Improve Environmental Education Innovation and Literacy in Waste Management: Bibliometric Research. Indonesian Journal on Learning and Advanced Education (IJOLAE), 3(2), 130–141. https://doi.org/10.23917/ijolae.v3i2.12782
Torbeyns, J., Verbruggen, S., & Depaepe, F. (2020). Pedagogical content knowledge in preservice preschool teachers and its association with opportunities to learn during teacher training. ZDM, 52(2), 269–280. https://doi.org/10.1007/s11858-019-01088-y
Valtonen, T., Sointu, E., Kukkonen, J., Mäkitalo, K., Hoang, N., Häkkinen, P., Järvelä, S., Näykki, P., Virtanen, A., Pöntinen, S., Kostiainen, E., & Tondeur, J. (2019). Examining pre‐service teachers’ Technological Pedagogical Content Knowledge as evolving knowledge domains: A longitudinal approach. Journal of Computer Assisted Learning, 35(4), 491–502. https://doi.org/10.1111/jcal.12353
Wang, C.-J. (2019). Facilitating the emotional intelligence development of students: Use of technological pedagogical content knowledge (TPACK). Journal of Hospitality, Leisure, Sport & Tourism Education, 25, 100198. https://doi.org/10.1016/j.jhlste.2019.100198
Wang, P., Coetzee, K., Strachan, A., Monteiro, S., & Cheng, L. (2020). Examining Rater Performance on the CELBAN Speaking: A Many-Facets Rasch Measurement Analysis. Canadian Journal of Applied Linguistics, 23(2), 73–95. https://doi.org/10.37213/cjal.2020.30436
Wang, P., Coetzee, K., Strachan, A., Monteiro, S., & Cheng, L. (2020). Examining rater performance on the CELBAN Speaking: A many-facets Rasch measurement analysis. Canadian Journal of Applied Linguistics, 23(2), 73–95. https://doi.org/10.37213/cjal.2020.30436
Wati, S. (2019). Psychometric validation of a video-based rubric for assessing pedagogical and professional competence of physics teacher candidates using Rasch analysis. Jurnal Inovasi Pendidikan IPA, 5(2), 234–245. https://doi.org/10.21831/jipi.v5i2.12345
Yusup, M. (2021). Using Rasch model for the development and validation of energy literacy assessment instrument for physics prospective teachers. Journal of Physics: Conference Series, 1876(1), 012056. https://doi.org/10.1088/1742-6596/1876/1/012056
Zeller, J., Schiering, D., Kulgemeyer, C., Neumann, K., Riese, J., & Sorge, S. (2024). Cross-project empirical and criteria-oriented analysis of physics prospective teachers’ pedagogical content knowledge: What content structures emerge in the context of different models? Teaching science. https://doi.org/10.1007/s42010-024-00200-w
Downloads
Published
How to Cite
Issue
Section
Citation Check
License
Copyright (c) 2025 Jurnal Inovasi Pendidikan IPA
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
The authors submitting a manuscript to this journal agree that, if accepted for publication, copyright publishing of the submission shall be assigned to Jurnal Inovasi Pendidikan IPA (JIPI). However, even though the journal asks for a copyright transfer, the authors retain (or are granted back) significant scholarly rights.
Jurnal Inovasi Pendidikan IPA by http://journal.uny.ac.id/index.php/jipi/index is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
