Development of digital literacy assessment instrument for prospective teacher students at state universities in the Special Region of Yogyakarta

Hari Kantona; Sudji Munadi

doi:10.21831/pep.v28i2.72712

Development of digital literacy assessment instrument for prospective teacher students at state universities in the Special Region of Yogyakarta

Hari Kantona, Universitas Negeri Yogyakarta, Indonesia
Sudji Munadi, Universitas Negeri Yogyakarta, Indonesia

10.21831/pep.v28i2.72712

Abstract

This research aims to (1) develop an assessment instrument for digital literacy skills of prospective teacher students, (2) describe the quality of the developed assessment instrument for digital literacy skills of prospective teacher students, and (3) depict the digital literacy skills of prospective teacher students based on the measurement results of the developed instrument.

The development of this research is carried out through 10 stages: (1) determining instrument specifications, (2) writing the instrument, (3) determining instrument scales, (4) determining scoring systems, (5) reviewing the instrument, (6) conducting limited-scale testing, (7) analyzing the instrument, (8) assembling the instrument, (9) conducting large-scale testing, and (10) interpreting the measurement results. Large-scale testing involves 262 prospective teacher students who have taken micro-teaching courses at Universitas Negeri Yogyakarta (UNY) and Universitas Islam Negeri (UIN) Sunan Kalijaga Yogyakarta. Construct validity is demonstrated through confirmatory factor analysis (CFA), while item characteristic information and the depiction of digital literacy skills of prospective teacher students are based on estimation results using the item response theory (IRT) approach with the graded response model (GRM). The reliability estimation of the instrument is demonstrated based on estimation results with the Cronbach's Alpha reliability coefficient.

This development study produces three main findings. (1) The instrument's construction is based on seven aspects with 35 statement items: seven items for information literacy, six items for digital scholarship, three items for learning skills, three items for ICT literacy, seven items for career and identity management, four items for communication and collaboration, and five items for media literacy. (2) The quality of the developed instrument is tested through content validity using Aiken’s V, construct validity using CFA, and item characteristic analysis using the IRT GRM model. (3) The depiction of digital literacy skills measurement among prospective teacher students at state universities in the Special Region of Yogyakarta is as follows: 9% categorized as excellent, 19% as good, 38% as sufficient, 30% as lacking, and 4% as very lacking.

Keywords

digital literacy, instruments, prospective student educators

References

Arifin, R., Kusumah, I. H., & Mubarak, I. (2018). Hasil Penilaian Diri Dan Penilaian Teman Sebaya Dibandingkan Dengan Assessment Dosen Untuk Hasil Produk Mata Kuliah Body Otomotif. Journal of Mechanical Engineering Education, 5(1), 79–80. https://doi.org/10.17509/jmee.v5i1.12623

Arifin, Z. (2017). Kriteria Instrumen dalam Suatu Penelitian. Jurnal THEOREMS (The Original Research of Mathematics), 2(1), 28–36.

Ayu, M. (2021). Implementasi Literasi Digital Pada Pembelajaran Fikih di MAN 1 Aceh Barat. UIN Syarif Hidayatullah Jakarta.

Bawden, D. (2001). Progress in documentation: Information and Digital Literacies a Review of Concepts. Journal of Documentation, 57(2), 230–251. https://doi.org/10.1108/eb026593

Creswell, J. W. (2012). Educational Research : Planning, Conducting, and Evaluating Quantitative and Qualitative Research / John W. Creswell. In University of Nebraska–Lincoln (4th ed.). Pearson Education Limited.

Falloon, G. (2020). From digital literacy to digital competence: the teacher digital competency (TDC) framework. Educational Technology Research and Development, 68(5), 2449. https://doi.org/10.1007/s11423-020-09767-4

Febliza, A., & Okatariani, O. (2020). Pengembangan Instrumen Literasi Digital Sekolah, Siswa Dan Guru. Jurnal Pendidikan Kimia Universitas Riau, 5(1), 1–9. https://doi.org/10.33578/jpk-unri.v5i1.7776

Fitriani, Yulis, P. A. R., Nurhuda, N., Sukarni, Gunita, A., & Cholijah, S. (2023). Peningkatan Literasi Digital Guru Dalam Pembelajaran Blended Learning. Jurnal Pengabdian UNDIKMA, 4(1), 140. https://doi.org/10.33394/jpu.v4i1.6536

Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2019). Multivariate Data Analysis. https://doi.org/10.1002/9781119409137.ch4

Hambleton, R. K., & Swaminathan, H. (1985). Item Response Theory: Principles and Applications. In G. F. Madaus & D. L. Stufflebeam (Eds.), Applied Psychological Measurement (1st ed.). Kluwer Nijhoff Publishing. https://doi.org/10.1007/978-94-017-1988-9

Hardini, R. T. (2020). Pengembangan Instrumen Penilaian Pendidikan Karakter untuk Siswa Sekolah Dasar. In UNY. https://medium.com/@arifwicaksanaa/pengertian-use-case-a7e576e1b6bf

Hidayat, R., & Abdillah. (2019). Ilmu Pendidikan. Lembaga Peduli Pengembangan Pendidikan Indonesia (LPPPI).

Istiyono, E. (2020). Pengembangan Instrumen Penilaian dan Analisis Hasil Belajar Fisika (2nd ed.). UNY Press.

Kemendikbud. (2017). Pedoman Penilaian dan Evaluasi Gerakan Literasi Nasional (L. A. Mayani (ed.)). Kementrian Pendidikan dan Kebudayaan. http://gln.kemdikbud.go.id/glnsite/wp-content/uploads/2017/08/pedoman-penilaian-evaluasi-gln.pdf

Kubiszyn, T. O. M., & Borich, G. (2013). Educational Testing & Measurement (A. Melhorn (ed.); 10th ed.). John Wiley & Sons, Inc.

Kurnianingsih, I., Rosini, R., & Ismayati, N. (2017). Upaya Peningkatan Kemampuan Literasi Digital Bagi Tenaga Perpustakaan Sekolah dan Guru di Wilayah Jakarta Pusat Melalui Pelatihan Literasi Informasi. Jurnal Pengabdian Kepada Masyarakat (Indonesian Journal of Community Engagement), 3(1), 62. https://doi.org/10.22146/jpkm.25370

Lankshear, C., & Knobel, M. (2008). Introduction: Digital Literacies-Concepts, Policies and Practices. In C. Lankshear & M. Knobel (Eds.), Digital Literacies (1st ed., pp. 2–4). Peter Lang Publishing. https://www.researchgate.net/publication/291334632

Liansari, V., & Nuroh, E. Z. (2018). Realitas Penerapan Literasi Digital bagi Mahasiswa FKIP Universitas Muhammadiyah Sidoarjo. Proceedings of The ICECRS, 1(3), 250–251. https://doi.org/10.21070/picecrs.v1i3.1397

Lukitasari, M., Murtafiah, W., Ramdiah, S., Hasan, R., & Sukri, A. (2022). Constructing Digital Literacy Instrument and its Effect on College Students’ Learning Outcomes. International Journal of Instruction, 15(2), 172. https://doi.org/10.29333/iji.2022.15210a

Magdalena, I., Ismawati, A., & Amelia, S. A. (2021). Penggunaan Evaluasi Non-Tes Dan Kesulitannya Di Sdn Gempol Sari. PENSA : Jurnal Pendidikan Dan Ilmu Sosial, 3(2), 187–199. https://ejournal.stitpn.ac.id/index.php/pensa

Magdalena, I., Maghfiroh, N., Yolawati, N. N., Dewanti, R. R., & Tangerang, U. M. (2021). Penggunaan Teknik Evaluasi Non Tes Pada Pembelajaran Ips Kelas Vi Di Sdn Selapajang Jaya2. 3(1), 113–123.

Mardapi, D. (2012). Pengukuran, Penilaian dan Evaluasi Pendidikan (1st ed.). Peter Lang Publishing.

Mardapi, D. (2017). Pengukuran Penilaian dan Evaluasi Pendidikan (2nd ed.). Parama Publishing.

Martin, A. (2008). Digital Literacy and the “Digital Society.” In C. Lankshear & M. Knobel (Eds.), Digital Literacies: Concepts, Policies and Practices (1st ed., pp. 156–173). Peter Lang Publishing.

Mills, K. A., Unsworth, L., & Scholes, L. (2022). Literacy for Digital Futures: Mind, Body, Text. In Literacy for Digital Futures: Mind, Body, Text. https://doi.org/10.4324/9781003137368

Naga, D. S. (1992). Pengantar Teori Sekor pada Pengukuran Pendidikan. Gunadarma.

Nasrullah, R., Aditya, W., Satya, T. I., Nento, M. N., Hanifah, N., Miftahussururi, & Akbari, Q. S. (2017). Gerakan Literasi Nasional. In L. A. Mayani (Ed.), Kementerian Pendidikan dan Kebudayaan. Kementerian Pendidikan dan Kebudayaan. https://gln.kemdikbud.go.id/glnsite/wp-content/uploads/2017/10/cover-materi-pendukung-literasi-finansial-gabung.pdf

Netemeyer, R. G., Boles, J. S., & McMurrian, R. (1996). Development and validation of work-family conflict and family-work conflict scales. Journal of Applied Psychology, 81(4), 405–406. https://doi.org/10.1037/0021-9010.81.4.400

Ng, D. T. K., Leung, J. K. L., Su, J., Ng, R. C. W., & Chu, S. K. W. (2023). Teachers’ AI digital competencies and twenty-first century skills in the post-pandemic world. Educational Technology Research and Development, 71(1), 137–161. https://doi.org/10.1007/s11423-023-10203-6

Nur, M. (2019). Literasi Digital Keagamaan Aktivis Organisasi Keagamaan Di Madrasah Aliyah Negeri (Man) Di Kota Bandung. Jurnal SMART (Studi Masyarakat, Religi, Dan Tradisi), 5(1), 1–13. https://doi.org/10.18784/smart.v5i1.745

Nurhayati, A. (2016). Prinsip dan tujuan penilaian tindakan kelas. Jurnal Pembelajaran, 5(1), 7.

Nurlailah. (2022). Analisis Literasi Digital dan Keterampilan Berpikir Kritis Mahasiswa pada Pembelajaran Berbasis Web [Universitas Islam Negeri Raden Intan Lampung]. In UIN Raden Intan Lampung. https://doi.org/10.30998/rdje.v8i1.11738

Organisation for Economic Co-operation and Development (OECD). (2019). Programme for International Student Assessment (PISA) Result from PISA 2018. https://doi.org/10.1007/978-94-6209-497-0_69

Prayitno, H. J., Sutama, Fuadi, D., Nurhidayat, Ratih, K., Harsono, Sufanti, M., Amalia, N., Sutarmi, S., Yuliana, I., & Hariyatmi. (2019). Desain dan Pedoman Pembelajaran Mikro (1st ed.). Muhammadiyah University Press.

Rahman, A. A., & Nasryah, C. E. (2019). Evaluasi Pembelajaran. In Uwais Inspirasi Indonesia (1st ed.). Uwais Inspirasi Indonesia. www.penerbituwais.com

Rahmawati. (2018). Analisis Kesenjangan Literasi Digital Mahasiswa Program Studi PAI PTKIN Indonesia [UIN Syarif Hidayatullah Jakarta]. In UIN Syarif Hidayatullah. http://repository.uinjkt.ac.id/dspace/handle/123456789/38121

Retnawati, H. (2014). Teori Respons Butir dan Penerapannya [Item Response Theory and its Application] (1st ed.). Nuha Medika.

Riel, J., Christian, S., & Hinson, B. (2012). Charting Digital Literacy: A Framework for Information Technology and Digital Skills Education in the Community College. SSRN Electronic Journal, 3–4. https://doi.org/10.2139/ssrn.2781161

Rukayah, Daryanto, J., Atmojo, I. R. W., Ardiansyah, R., Saputri, D. Y., & Salimi, M. (2022). Augmented Reality Media Development in STEAM Learning in Elementary Schools. Ingenierie Des Systemes d’Information, 27(3), 463. https://doi.org/10.18280/isi.270313

Sánchez-Cruzado, C., Santiago Campión, R., & Sánchez-Compaña, M. T. (2021). Teacher digital literacy: The indisputable challenge after covid-19. Sustainability (Switzerland), 13(4), 1–29. https://doi.org/10.3390/su13041858

Setyaningsih, R., Abdullah, A., Prihantoro, E., & Hustinawaty, H. (2019). Model Penguatan Literasi Digital Melalui Pemanfaatan E-Learning. Jurnal ASPIKOM, 3(6), 1202. https://doi.org/10.24329/aspikom.v3i6.333

Soeparno, P. L., & Ismaniati, C. (2022). Mathematics Pre-Service Teachers’ Digital Literacy and Their Readiness Towards 21St Century Learning: a Mixed Method Study. BAREKENG: Jurnal Ilmu Matematika Dan Terapan, 16(2), 646–648. https://doi.org/10.30598/barekengvol16iss2pp643-650

Susilowati, & Estidarsani, N. (2017). Penerapan Self Assessment (Penilaian Diri) dengan Rubrik pada Hasil Belajar Siswa Menggambar Konstruksi Kusen Pintu Dan Jendela di Kelas X TGB SMKN 1 Kemlagi, Mojokerto. Jurnal Kajian Pendidikan Teknik Bangunan, 3, 243. tekniksipilunesa.org

Umar, J., & Nisa, Y. F. (2020). Uji Validitas Konstruk dengan CFA dan Pelaporannya. Jurnal Pengukuran Psikologi Dan Pendidikan Indonesia, 9(2), 7–8. https://doi.org/10.15408/jp3i.v9i2.16964

Wahyuningsih, S. (2021). Modul Literasi Digital Di Sekolah Dasar. Kementerian Pendidikan, Kebudayaan, Riset, Dan Teknologi, 1.

Warsiyah, Madrah, M., Muflihin, A., & Irfan, A. (2022). Urgensi Literasi Digital bagi Pendidik dalam Meningkatkan Keterampilan Mengelola Pembelajaran. Dimas: Jurnal Pemikiran Agama Untuk Pemberdayaan, 22(1), 124–126. https://doi.org/10.21580/dms.2022.221.10042

Yanti, N., Mulyati, Y., Sunendar, D., & Damaianti, V. (2021). Tingkat Literasi Digital Mahasiswa Indonesia. Diksa : Pendidikan Bahasa Dan Sastra Indonesia, 7(1), 61–63. https://doi.org/10.33369/diksa.v7i1.22391

Yunita, I., & Indrawati, I. (2021). Open Access jurnal dalam meningkatkan kemampuan literasi digital. Publishing Letters, 1(2), 12–13. https://doi.org/10.48078/publetters.v1i2.20

Yusuf, M. (2018). Pengantar Ilmu Pendidikan. In Lembaga Penerbit Kampus IAIN Palopo.

DOI: https://doi.org/10.21831/pep.v28i2.72712