Developing and Validating a Rubric for Measuring Skills in Designing Science Experiments for Prospective Science Teachers

Rizki Nor Amelia; Prasetyo Listiaji; Novi Ratna Dewi; Andhina Putri Heriyanti; Bagus Dwi Atmaja; Tafuz Mahabatis Shoba; Imam Sajidi

doi:10.21831/jipi.v10i1.65853

Developing and Validating a Rubric for Measuring Skills in Designing Science Experiments for Prospective Science Teachers

Rizki Nor Amelia, Universitas Negeri Semarang, Indonesia
Prasetyo Listiaji, Universitas Negeri Semarang, Indonesia
Novi Ratna Dewi, Universitas Negeri Semarang, Indonesia
Andhina Putri Heriyanti, Universitas Negeri Semarang, Indonesia
Bagus Dwi Atmaja, Universitas Negeri Semarang, Indonesia
Tafuz Mahabatis Shoba, Universitas Negeri Semarang, Indonesia
Imam Sajidi, Universitas Negeri Semarang, Indonesia

10.21831/jipi.v10i1.65853

Abstract

The Indonesian government mandates that science teachers must have competence in designing science experiments for learning purposes so that science content can be learned optimally by students while preparing them to have the ability to face the 21st century. This is development research that aims to develop a measurement instrument for science experiment design skills for prospective science teachers that meets good psychometric characteristics. The rubric development procedure refers to the Churches rubric development method, which consists of four stages: define, design, do, and debrief, involving 10 experts (lecturers and teachers) and 124 prospective science teachers as research participants. The results of exploratory and confirmatory factor analysis showed that the analytical rubric developed by measuring ten aspects, namely title, research objectives, relevant theories, variables, materials, equipment and instrumentation, method, an appropriate number of data, references, and systematic and technical writing was valid in content (CVI=.96), valid in construct (GFI=.94; RMSEA=.071; NFI=.99; CFI=1.00; PNFI=.91), and reliable (α=.968). The use of a standardized rubric certainly allows the assessment to provide consistent, accurate, and objective results and helps students understand what competencies they must achieve.

Keywords

prospective science teacher; rubric; science experiment design skills

Full Text:

FULLTEXT PDF

References

Abd Rauf, R. A., Rasul, M. S., Mans, A. N., Othman, Z., & Lynd, N. (2013). Inculcation of science process skills in a science classroom. Asian Social Science, 9(8), 1911-2017. https://doi.org/10.5539/ass.v9n8p47

Afthanorhan, A., Awang, Z., & Aimran, N. (2020). An extensive comparison of CB-SEM and PLS-SEM for reliability and validity. International Journal of Data and Network Science, 4(4), 357-364. https://doi.org/10.5267/j.ijdns.2020.9.003

Albright, J. J., & Park, H. M. (2006). Confirmatory factor analysis using AMOS, LISREL, and MPLUS. Bloomington, IN: The Trustees of Indiana University

Andrade, H., & Du, Y. (2005). Student perspectives on rubric-referenced assessment. Practical Assessment, Research & Evaluation, 10, 1–11. https://doi.org/10.1080/02602930801955986

Arribas, M. (2004). Diseño y validación de cuestionarios. Matronas profesión, 5(17), 23-29.

Aulia, V., & Yamin, M. (2020). Students’ activities in integrated thematic textbooks for primary school to meet 21st century skills. Paedagoria: Jurnal Kajian, Penelitian dan Pengembangan Kependidikan, 11(3), 273-282.

Bahtiar, Maimun, & Anggruani, B. L. (2022). Pengaruh model discovery learning melalui kegiatan praktikum IPA Terpadu terhadap kemampuan berpikir kritis siswa. Jurnal Pendidikan MIPA, 12(2), 134–142. https://doi.org/10.37630/jpm.v12i2.564

Chan, L. L., & Idris, N. (2017). Validity and reliability of the instrument using exploratory factor analysis and Cronbach’s alpha. International Journal of Academic Research in Business and Social Sciences, 7(10), 400-410.

Cheung, G. W., Cooper-Thomas, H. D., Lau, R. S., & Wang, L. C. (2023). Reporting reliability, convergent and discriminant validity with structural equation modeling: A review and best-practice recommendations. Asia Pac J Manag, 1-39 https://doi.org/10.1007/s10490-023-09871-y

Chowdhury, F. (2019). Application of rubrics in the classroom: A vital tool for improvement in assessment, feedback and learning. International education studies, 12(1), 61-68. https://doi.org/10.5539/ies.v12n1p61

Churches, A. (2015). A guide to formative and summative assessment and rubric development. 21st Century Project

Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297-324.

Darmaji, D., Astalini, A., Kurniawan, D. A., & Triani, E. (2022). The effect of science process skills of students argumentation skills. Jurnal Inovasi Pendidikan IPA, 8(1), 78-88. https://doi.org/10.21831.jipi.v8i1.49224

Darnita, I. K, Marhaeni, A. A. I. N., & Candiasa, M. (2014). Pengaruh penggunaan bahan ajar online terhadap prestasi belajar tikom dengan kovariabel aktivitas belajar siswa kelas VIII SMP Dwijendra Gianyar. e-Journal Program Pascasarjana Universitas Pendidikan Ganesha, 4, 1-10

Dash, G., & Paul, J. (2021). CB-SEM vs PLS-SEM methods for research in social sciences and technology forecasting. Technological Forecasting and Social Change, 173, 1-11. https://doi.org/10.1016/j.techfore.2021.121092

Dewi, N. L. P. E. S. (2022, December). Self-assessment checklist for assessing young learners’ writing performance. In 2nd International Conference on Languages and Arts across Cultures (ICLAAC 2022) (pp. 96-101). Atlantis Press. https://doi.org/10.2991/978-2-494069-29-9_11

Fernandes, A. A. F., & Solimun, P. E. (2005). Kajian korelasi antar measurement error pada analisis struktural equation model. Malang: Universitas Brawijaya

Fiteriani, I. (2017). Studi komparasi perbedaan pengaruh pemahaman konsep dan penguasaan keterampilan proses sains terhadap kemampuan mendesain eksperimen sains. Terampil: Jurnal Pendidikan dan Pembelajaran Dasar, 4(1). 47-80

Gebremedhin, M., Gebrewahd, E., & Stafford, L. K. (2022). Validity and reliability study of clinician attitude towards rural health extension program in Ethiopia: exploratory and confirmatory factor analysis. BMC Health Services Research, 22(1), 1-10. https://doi.org/10.1186/s12913-022-08470-9

Gehlbach, H., & Brinkworth, M. E. (2011). Measure twice, cut down error: A process for enhancing the validity of survey scales. Review of general psychology, 15(4), 380-387.

Geldhof, G. J., Preacher, K. J., & Zyphur, M. J. (2014). Reliability estimation in a multilevel confirmatory factor analysis framework. Psychological Methods, 19(1), 72–91. https://doi.org/10.1037/a0032138

Ghazali, N., Nordin, M. S., Hashim, S., & Hussein, S. (2017, October). Measuring content validity: Students’ self-efficacy and meaningful learning in massive open online course (MOOC) scale. In International Conference on Education in Muslim Society (ICEMS 2017) (pp. 128-133). Atlantis Press.

Grewal, R., Cote, J. A., & Baumgartner, H. (2004). Multicollinearity and measurement error in structural equation models: Implications for theory testing. Marketing Science, 23, 519–529.

Gürses, A., Çetinkaya, S., Doğar, Ç., & Şahin, E. (2015). Determination of levels of use of basic process skills of high school students. Procedia-Social and Behavioral Sciences, 191, 644-650. https://doi.org/10.1016/j.sbspro.2015.04.243

Güvendir, M. A., & Özkan, Y. Ö. (2022). Item removal strategies conducted in exploratory factor analysis: A comparative study. International Journal of Assessment Tools in Education, 9(1), 165-180. https://doi.org/10.21449/ijate.827950

Hair Jr., J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2009). Multivariate data analysis (7th Ed). Upper Saddle River: Prentice Hall

Hanike, Y., & Damirah, D. (2018). Modifikasi model analisis structural equation model (SEM) pada reaksi pasar di perusahaan bursa efek Indonesia melalui modification indices. Matematika dan Pembelajaran, 6(2), 127-142.

Hauben, M., Hung, E., & Hsieh, W. Y. (2017). An exploratory factor analysis of the spontaneous reporting of severe cutaneous adverse reactions. Therapeutic advances in drug safety, 8(1), 4-16. https://doi.org/10.1177/ 2042098616670799

Hidayat, R., Zamri, S. N. A. S., & Zulnaidi, H. (2018). Exploratory and confirmatory factor analysis of achievement goals for Indonesian students in mathematics education programmes. EURASIA Journal of Mathematics, Science and Technology Education, 14(12), 1-12. https://doi.org/10.29333/ejmste/99173

Hofstein, A., & Mamlok-Naaman, R. (2007). The laboratory in science education: the state of the art. Chemistry Education Research and Practice, 8(2), 105-107. https://doi.org/10.1039/B7RP90003A

Hooper, D., Coughlan, J. & Mullen, M. R. (2008). Structural equation modelling: guidelines for determining model fit. The Electronic Journal of Business Research Methods, 6(1), 53 - 60

Iriani, T., Anisah, Y. L., Maknun, J., & Dewi, N. I. K. (2023, May). Analytical rubric development design for objective test assessment. In ACEIVE 2022: Proceedings of the 4th Annual Conference of Engineering and Implementation on Vocational Education, ACEIVE 2022, 20 October 2022, Medan, North Sumatra, Indonesia (p.327). European Alliance for Innovation. https://doi.org/10.4108/eai.20-10-2022.2328882

Isbell, T., & Goomas, D. T. (2014). Computer-assisted rubric evaluation: Enhancing outcomes and. assessment quality. Community College Journal of Research and Practice, 38(12), 1193-1197. https://doi.org/10.1080/10668926.2014.899526

Jayanti, M. I., & Nurfathurrahmah. (2023). Gerakan penguatan literasi sains melalui praktikum ipa sederhana di SMPN 11 Kota Bima. Taroa: Jurnal Pengabdian Masyarakat, 2(1), 1–8. https://doi.org/10.52266/taroa.v2i1.1220

Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28(4), 563–575. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x

Ledesma, R. D., Valero-Mora, P., & Macbeth, G. (2015). The scree test and the number of factors: a dynamic graphics approach. The Spanish journal of psychology, 18, 1-10. https://doi.org/10.1017/sjp.2015.13

Lestari, P., Ristanto, R. H., & Miarsyah, M. (2019). Metacognitive and conceptual understanding of pteridophytes: Development and validity testing of an integrated assessment tool. Indonesian Journal of Biology Education, 2(1), 15-24. https://doi.org/10.31002/ijobe.v2i1.1225.

Maiyanti, S. I., Dwipurwani, O., Desiani, A., & Aprianah, B. (2008). Aplikasi analisis faktor konfirmatori untuk mengetahui hubungan peubah indikator dengan peubah laten yang mempengaruhi prestasi mahasiswa di Jurusan Matematika FMIPA UNSRI. Jurnal Pendidikan Matematika, 2(1), 15-30.

Malik, A., Aliah, H., & Susanti, S. (2019). Peran praktikum dalam pembelajaran IPA. Bandung: Pusat Penelitian dan Penerbitan UIN SGD Bandung

McDonald, R. P., & Ho, M. H. R. (2002). Principles and practice in reporting statistical equation analyses. Psychol. Methods, 7 (1), 64–82.

Muhammad, A., Lebar, O., & Mokshein, S. E. (2018). Rubrics as assessment, evaluation and scoring tools. International Journal of Academic Research in Business and Social Sciences, 8(10), 1417–1431

Nugraha, M. G., Suhandi, A., & Rusnayati, H. (2022). Meningkatkan kompetensi guru SMA/MA dalam mendesain eksperimen fisika sebagai upaya melatihkan keterampilan abad 21. WaPFi: Wahana Pendidikan Fisika, 7(1), 91-97

Kemendikbud RI. (2017). Peraturan Menteri Pendidikan Nasional Republik Indonesia Nomor 16 Tahun 2017 tentang Standar Kualifikasi Akademik dan Kompetensi Guru

Kurniawati, A. (2021). Science process skills and its implementation in the process of science learning evaluation in schools. Journal of Science Education Research, 5(2), 16-20. https://doi.org/10.21831/jser.v5i2.44269

Ongowo, R.O & Indoshi, F.C. (2013). Science process skill in Kenya certificate of secondary education biology practical examination. Journal of Scientific research, 4(11): 713-717

Pino, M. E. M., Ordoñez, F. R. R., Ysa, R. A. S., Llanos, D. M. J., Cruz, M. M. T., & Calderón, B. A. C. (2023). Role of expert in validation of information collection instruments for business purposes. International Journal of Professional Business Review, 8(8), 1-11. https://doi.org/10.26668/businessreview/2023.v8i8.3122

Pratiwi, U., Akhdinirwanto, R. W., Fatmaryanti, S. D., & Ashari, A. (2020). Penerapan metode eksperimen materi Gerak Lurus Berubah Beraturan (GLBB) pada kegiatan praktikum fisika dasar untuk meningkatkan sikap ilmiah siswa MA Al-Iman Bulus Purworejo. Surya Abdimas, 4(1), 1–7. https://doi.org/10.37729/abdimas.v4i1.413

Purnamasari, S. (2020). Pengembangan praktikum IPA terpadu tipe webbed untuk meningkatkan keterampilan proses sains. Pancasakti Science Education Journal, 5(2), 8–15. https://doi.org/10.24905/psej.v5i2.20

Pursitasari, I. D., Permanasari, A., Rubini, B., Ardianto, D., Heliawati, L., Nulhakim, L., Kurniasih, S., & Taufik, A. N. (2023). Pelatihan penyusunan desain praktikum dan penggunaan KIT praktikum IPA bagi guru IPA SMP di Kabupaten Serang. Jurnal ABDINUS: Jurnal Pengabdian Nusantara, 7(2), 516-530. https://doi.org/10.29407/ja.v7i2.19495

Raykov, T., Goldammer, P., Marcoulides, G. A., Li, T., & Menold, N. (2018). Reliability of scales with second-order structure: Evaluation of coefficient alpha’s population slippage using latent variable modeling. Educational and Psychological Measurement, 78, 1123–1135.

Reddy, Y. M., & Andrade, H. (2010). A review of rubric use in higher education. Assessment and Evaluation in Higher Education, 35(4), 435–448. https://doi.org/10.1080/02602930902862859

Retnawati, H. (2017). Validitas dan reliabilitas konstruk skor tes kemampuan calon mahasiswa. Jurnal Ilmu Pendidikan, 23(2), 126 -135. https://doi.org/10.17977/jip.v23i2.10973

Ristanto, R. H., Zubaidah, S., Amin, M., & Rohman, F. (2018). From a reader to a scientist: developing cirgi learning to empower scientific literacy and mastery of biology concept. Biosfer: Jurnal Pendidikan Biologi, 11(2), 90-100. https://doi.org/10.21009/biosferjpb.v11n2.90-100

Rivas, M. R., De La Serna, M. C., & Martinez-Figueira, E. (2014). Electronic rubrics to assess competences in ICT subjects. European Educational Research Journal, 13(5), 584–594. https://doi.org/10.2304/eerj.2014.13.5.584

Rukmini, D., & Saputri, L. (2017). The authentic assessment to measure students' English productive skills based on 2013 Curriculum. Indonesian Journal of Applied Linguistics, 7(2), 263–273. https://doi.org/10.17509/ijal.v7i2.8128

Sadler, D. R. (2009). Indeterminacy in the use of preset criteria for assessment and grading. Assessment & Evaluation in Higher Education, 34(2), 159–179. https://doi.org/10.1080/02602930801956059

Sari, P. M., & Zulfadewina. (2020). Pengembangan panduan praktikum berbasis keterampilan proses sains pada mata kuliah praktikum IPA SD. Jurnal Pelita Pendidikan, 8(1), 94–98. https://doi.org/10.24114/jpp.v8i1.17334

Shana, Z., & Abulibdeh, E. S. (2020). Science practical work and its impact on high students' academic achievement. JOTSE, 10(2), 199-215. https://doi.org/10.3926/jotse.888

Shrestha, N. (2021). Factor analysis as a tool for survey analysis. American Journal of Applied Mathematics and Statistics, 9(1), 4-11.

Stevens, D. D., & Levi, A. J. (2013). Introduction to rubrics (2nd Ed.). Sterling, VA: Stylus Publishing

Sudaryanto, M., & Akbariski, H. S. (2021). Students' competence in making language skill assessment rubric. Research and Evaluation in Education, 7(2), 156-167. https://doi.org/10.21831/reid.v7i2.44005

Sujati, H., Sajidan, Akhyar, M., & Gunarhadi, (2020). Testing the construct validity and reliability of curiosity scale using confirmatory factor analysis. Journal of Educational and Social Research, 20(4). 229-237.

Suyatno, M., Rukhmana, T., Nurmiati, A. S., Romadhon, F., Irawan, I., & Mokodenseho, S. (2023). Penerapan kurikulum merdeka sebagai upaya dalam mengatasi krisis pembelajaran (learning loss) pada mata pelajaran pendidikan agama islam kelas x di SMA Negeri 12 Bandar Lampung. Journal of Education, 6(1), 3588-3600.

Turiman, P., Omar, J., Daud, A. M., & Osman, K. (2012). Fostering the 21st century skills through scientific literacy and science process skills. Procedia-Social and Behavioral Sciences, 59, 110-116.

Vinas, L.F. (2022). Testing the Reliability of two rubrics used in official english certificates for the assessment of writing. Alicante Journal of English Studies, 36, 85-109. https://doi.org/10.14198/raei.2022.36.05

Wola, B. R., Rungkat, J. A., & Harindah, G. M. D. (2023). Science process skills of prospective science teachers' in practicum activity at the laboratory. Jurnal Inovasi Pendidikan IPA, 9(1), 50-61. https://doi.org/10.21831/jipi.v9i1.52974

Wu, R. M., Zhang, Z., Zhang, H., Wang, Y., Shafiabady, N., Yan, W., Gou, J., Gide, E., & Zhang, S. (2023). An FSV analysis approach to verify the robustness of the triple-correlation analysis theoretical framework. Scientific Reports, 13(1), 1-20. https://doi.org/10.1038/s41598-023-35900-3

Xia, Y., & Yang, Y. (2019). RMSEA, CFI, and TLI in structural equation modeling with ordered categorical data: The story they tell depends on the estimation methods. Behavior research methods, 51, 409-428. 8 https://doi.org/10.3758/s13428-018-1055-2

Yuanita, Y., & Yuniarita, F. (2018, November). Analysis of student's science practicum worksheet component of elementary school teachers in Gerunggang. In Profunedu International Conference Proceeding (Vol. 1, pp. 341-346)

DOI: https://doi.org/10.21831/jipi.v10i1.65853