NGSS-oriented chemistry test instruments: Validity and reliability analysis with the Rasch model

Roudloh Muna Lia; Ani Rusilowati; Wiwi Isnaeni

doi:10.21831/reid.v6i1.30112

NGSS-oriented chemistry test instruments: Validity and reliability analysis with the Rasch model

Roudloh Muna Lia, Universitas Negeri Semarang, Indonesia
Ani Rusilowati, Universitas Negeri Semarang, Indonesia
Wiwi Isnaeni, Universitas Negeri Semarang, Indonesia

10.21831/reid.v6i1.30112

Abstract

The instrument of measuring test attributes must be valid and reliable. This study was carried out since the validity and reliability testing of the chemistry items used by the testee is necessary. This study aims to estimate the validity and determine the reliability of chemical test instruments oriented Next Generation Science Standards (NGSS). The research was conducted through a quantitative descriptive approach in two vocational schools of engineering program which had 130 testees. The instrument used was an NGSS-oriented chemistry test instrument containing 35 items and an expert validation questionnaire. The obtained test participant's response from the test instrument was collected through the documentation method. Item in NGSS test were presented to three subject matters experts. The validities used were the content validity and the construct validity. The reliability was tested through internal consistency and interrater consistency approaches. The results show that content validity (Aiken’s V) is at a range of 0.50 to 1.00. The value of the unexplained variance is less than 10%, which means that it is well-categorized. This analysis is strengthened by CFA which has a goodness of fit and a good measurement model fit. The parameters used to test model fit are CFI, NFI, RMSEA and the value of loading factor. Some results values are over 0.90 and RMSEA is 0.00 and more than 0.3 of loading factor value on each item. All scales had alpha reliability more than the criteria of 0.70. Thus, the developed chemical test item were proven as valid and reliable instruments.

Keywords

validity; reliability; NGSS

Full Text:

PDF

References

Amalia, N. F., & Susilaningsih, E. (2014). Pengembangan instrumen penilaian keterampilan berpikir kritis siswa SMA pada materi asam basa. Jurnal Inovasi Pendidikan Kimia, 8(2), 1280–1389. Retrieved from https://journal.unnes.ac.id/nju/index.php/JIPK/article/view/4443

Ardiyanti, D. (2016). Aplikasi model Rasch pada pengembangan skala efikasi diri dalam pengambilan keputusan karir siswa. Jurnal Psikologi, 43(3), 248–263. https://doi.org/10.22146/jpsi.17801

Arifin, W. N., Yusoff, M. S. B., & Naing, N. N. (2012). Confirmatory factor analysis (CFA) of USM Emotional Quotient Inventory (USMEQ-i) among medical degree program applicants in Universiti Sains Malaysia (USM). Education in Medicine Journal, 4(2), 1–22. https://doi.org/10.5959/eimj.v4i2.33

Astuti, R., Sunarno, W., & Sudarisman, S. (2016). Pembelajaran IPA dengan pendekatan ketrampilan proses sains menggunakan metode Eksperimen Bebas Termodifikasi dan Eksperimen Terbimbing ditinjau dari sikap ilmiah dan motivasi belajar siswa. Proceeding Biology Education Conference, 13(1), 338–345. Retrieved from https://jurnal.uns.ac.id/prosbi/article/view/5742

Banne, K. (2018). Meningkatkan aktivitas belajar kimia (Redoks) siswa kelas XII TKR SMK Negeri 1 Sumarorong melalui penerapan model pembelajaran kooperatif tipe NHT dengan materi berbasis kontekstual. Jurnal MEKOM (Media Komunikasi Pendidikan Kejuruan), 5(1), 45–50. https://doi.org/10.26858/mekom.v5i1.8223

Bhakti, Y. B. (2015). Pengaruh jumlah alternatif jawaban dan teknik penskoran terhadap reliabilitas tes. Formatif: Jurnal Ilmiah Pendidikan MIPA, 5(1), 1–13. https://doi.org/10.30998/formatif.v5i1.168

Damelin, D. (2017). Using technology to enhance NGSS-aligned assessment tasks for classroom formative use. Retrieved from The Concord Consortium website: https://concord.org/newsletter/2017-spring/using-technology-enhance-ngss-aligned-assessment-tasks/

Dewi, P. C. P., & Sukadiyanto, S. (2015). Pengembangan tes keterampilan olahraga woodball untuk pemula. Jurnal Keolahragaan, 3(2), 228–240. https://doi.org/10.21831/jk.v3i2.6254

Dockrell, S., O’Grady, E., Bennett, K., Mullarkey, C., Mc Connell, R., Ruddy, R., … Flannery, C. (2012). An investigation of the reliability of Rapid Upper Limb Assessment (RULA) as a method of assessment of children’s computing posture. Applied Ergonomics, 43(3), 632–636. https://doi.org/10.1016/j.apergo.2011.09.009

Government Regulation No. 32 of 2013, on National Education Standard. , (2013).

Hakiki, A. W., Fitri, A. R., & Agung, I. M. (2018). Analisis properti psikometri subtes Merkaufgaben (ME) dengan Rasch model. Jurnal Psikologi, 14(1), 40–49. https://doi.org/10.24014/jp.v14i1.4900

Hasnah, H. (2017). Analisis kualitas soal matematika Ujian Sekolah kelas XII IPA SMA Negeri di Watansoppeng berdasarkan Teori Respon Butir. PEP Educational Assessment, 1(1), 27–33. Retrieved from https://ojs.unm.ac.id/UEA/article/view/3776

Hayati, S., & Lailatussaadah, L. (2016). Validitas dan reliabilitas instrumen pengetahuan pembelajaran aktif, kreatif dan menyenangkan (PAKEM) menggunakan model Rasch. Jurnal Ilmiah Didaktika, 16(2), 169–179. https://doi.org/10.22373/jid.v16i2.593

Iskandar, A. (2017). Teknik analisis validitas konstruk dan reliabilitas instrument test dan non test dengan software LISREL. https://doi.org/10.31227/osf.io/nbhxq

Ismail, I., Permanasari, A., & Setiawan, W. (2016). STEM virtual lab: An alternative practical media to enhance student’s scientific literacy. Jurnal Pendidikan IPA Indonesia, 5(2), 239–246. https://doi.org/10.15294/jpii.v5i2.5492

Kadir, A. (2015). Menyusun dan menganalisisi tes hasil belajar. AL-TA’DIB : Jurnal Kajian Ilmu Kependidikan, 8(2), 70–81. https://doi.org/10.31332/atdb.v8i2.411

Khumaedi, M. (2012). Reliabilitas instrumen penelitian pendidikan. Jurnal Pendidikan Teknik Mesin, 12(1), 25–30. Retrieved from https://journal.unnes.ac.id/nju/index.php/JPTM/article/view/5273

Kusaeri, K., Sutini, S., Suparto, S., & Wardah, F. (2019). The validity and inter-rater reliability of project assessment in mathematics learning. Beta: Jurnal Tadris Matematika, 12(1), 1–13. https://doi.org/10.20414/betajtm.v12i1.266

Lia, R. M. (2019). Pengembangan butir soal Kimia berorientasi NGSS dan analisisnya menggunakan model Rasch. Master thesis, Universitas negeri Semarang, Semarang.

Lia, R. M., & Isnaeni, I. (2018). Evaluation of Chemistry learning programs at vocational high school Semarang on Vehicle Engineering field. Proceedings of the International Conference on Science and Education and Technology 2018 (ISET 2018), 403–407. https://doi.org/10.2991/iset-18.2018.82

Linacre, J. M. (2016). A user’s guide to WINSTEPS MINISTEP Rasch-model computer programs. Chicago, IL: Winsteps.com.

Mohamad, M. M., Sulaiman, N. L., Sern, L. C., & Salleh, K. M. (2015). Measuring the validity and reliability of research instruments. Procedia - Social and Behavioral Sciences, 204, 164–171. https://doi.org/10.1016/j.sbspro.2015.08.129

National Research Council. (2013). Next Generation Science Standards: For states, by states. https://doi.org/10.17226/18290

Nurcahyo, F. A. (2016). Aplikasi IRT dalam analisis aitem tes kognitif. Buletin Psikologi, 24(2), 64–75. https://doi.org/10.22146/buletinpsikologi.25218

Othman, N. B., Salleh, S. M., Hussein, H., & Wahid, H. B. A. (2014). Assessing construct validity and reliability of competitiveness scale using Rasch model approach. The 2014 WEI International Academic Conference Proceedings, 113–120. Retrieved from https://www.westeastinstitute.com/wp-content/uploads/2014/06/Suria-Mohd-Salleh.pdf

Pancoro, N. H. (2011). Karakteristik butir soal ulangan kenaikan kelas sebagai persiapan bank soal Bahasa Inggris. Jurnal Penelitian Dan Evaluasi Pendidikan, 15(1), 92–114. https://doi.org/10.21831/pep.v15i1.1089

Penuel, W. R., Harris, C. J., & DeBarger, A. H. (2015). Implementing the Next Generation Science Standards. Phi Delta Kappan, 96(6), 45–49. https://doi.org/10.1177/0031721715575299

Pinilih, F. W., Budiharti, R., & Ekawati, E. Y. (2013). Pengembangan instrumen penilaian produk pada pembelajaran IPA untuk siswa SMP. Jurnal Pendidikan Fisika, 1(2), 23–27. Retrieved from https://jurnal.fkip.uns.ac.id/index.php/pfisika/article/view/2798

Prabowo, A., & Ristiani, E. (2011). Rancang bangun instrumen tes kemampuan keruangan pengembangan tes kemampuan keruangan Hubert Maier dan identifikasi penskoran berdasar teori Van Hielle. Kreano, Jurnal Matematika Kreatif-Inovatif, 2(2), 72–87. https://doi.org/10.15294/kreano.v2i2.2618

Reise, S. P., Widaman, K. F., & Pugh, R. H. (1993). Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance. Psychological Bulletin, 114(3), 552–566. https://doi.org/10.1037/0033-2909.114.3.552

Reiser, B. J. (2013). What professional development strategies are needed for successful implementation of the Next Generation Science Standards. The Invitational Research Symposium on Science Assessment, 1–23. Retrieved from http://www.ets.org/Media/Research/pdf/reiser.pdf

Reynolds, C. R., Livingston, R. B., & Willson, V. L. (2010). Measurement and assessment in education (2nd ed.). Upper Saddle River, NJ: Pearson Education.

Rusilowati, A. (2014). Pengembangan instrumen penilaian. Semarang: Unnes Press.

Sabekti, A. W., & Khoirunnisa, F. (2018). Penggunaan Rasch model untuk mengembangkan instrumen pengukuran kemampuan berpikir kritis siswa pada topik ikatan kimia. Jurnal Zarah, 6(2), 68–75. https://doi.org/10.31629/zarah.v6i2.724

Sohail, M. S., & Jang, J. (2017). Understanding the relationships among internal marketing practices, job satisfaction, service quality and customer satisfaction: An empirical investigation of Saudi Arabia’s service employees. International Journal of Tourism Sciences, 17(2), 67–85. https://doi.org/10.1080/15980634.2017.1294343

Sudrajat, D. (2016). Portofolio: Sebuah model penilaian dalam Kurikulum Berbasis Kompetensi. Intelegensia, 1(2), 1–9. Retrieved from http://ejurnal.unikarta.ac.id/index.php/intelegensia/article/view/257

Sumintono, B., & Widhiarso, W. (2015). Aplikasi pemodelan Rasch pada assessment pendidikan. Cimahi: Trim Komunikata.

Utami, B. N. (2018). Praktik evaluasi penyuluhan pertanian. Malang.

Wibisono, S. (2014). Aplikasi model Rasch untuk validasi instrumen pengukuran fundamentalisme agama bagi responden muslim. JP3I (Jurnal Pengukuran Psikologi Dan Pendidikan Indonesia), 3(3), 729–750. https://doi.org/10.15408/jp3i.v3i3.10731

Zehir, C., Akyuz, B., Eren, M. S., & Turhan, G. (2013). The indirect effects of servant leadership behavior on organizational citizenship behavior and job performance: Organizational justice as a mediator. International Journal of Research in Business and Social Science (2147-4478), 2(3), 1–13. https://doi.org/10.20525/ijrbs.v2i3.68

DOI: https://doi.org/10.21831/reid.v6i1.30112