A study of the readiness of post-pandemic computer-based four-tier diagnostic test (CBFTDT): A review of economic level, school grades, and device accessibility

Edi Istiyono; Wipsar Sunu Brams Dwandaru; Made Rai Shanti Ayub; Duden Saepuzaman; Rizki Zakwandi; Anisyah Rachman; Eri Yusron; Purwoko Haryadi Santoso

doi:10.21831/pep.v27i1.56005

A study of the readiness of post-pandemic computer-based four-tier diagnostic test (CBFTDT): A review of economic level, school grades, and device accessibility

Edi Istiyono, Universitas Negeri Yogyakarta, Indonesia
Wipsar Sunu Brams Dwandaru, Universitas Negeri Yogyakarta, Indonesia
Made Rai Shanti Ayub, Universitas Kristen Satya Wacana, Indonesia
Duden Saepuzaman, Universitas Pendidikan Indonesia, Indonesia
Rizki Zakwandi, Universitas Pendidikan Indonesia, Indonesia
Anisyah Rachman, Universitas Negeri Yogyakarta, Indonesia
Eri Yusron, Universitas Negeri Yogyakarta, Indonesia
Purwoko Haryadi Santoso, Universitas Negeri Yogyakarta, Indonesia

10.21831/pep.v27i1.56005

Abstract

Student readiness for using computer-based assessment was affected by economic level, school grade, and device accessibility. This study analyzed student readiness using a computer-based assessment, the four-tier diagnostic test (CBFTDT). The data in this study were students' responses to a questionnaire consisting of 16 statements about three aspects: mental access, skill access, and usage. The questionnaire consists of 16 statements with three aspects: items, media, and effectivity. The questionnaire was proved valid by experts and its estimated reliability score was 0.99. This study proved that the Student is still not ready to use the computer-based assessment, with an average readiness of 60%. In addition, there is no effect of economic level, school grade, and device accessibility on the readiness for using the computer-based assessment. Hence, the treatment of all stockholders is needed to prepare students to use the computer-based assessment.

Keywords

computer-based test; four-tier diagnostic test; student readiness

Full Text:

PDF

References

Ahmad, A., Al-Mashari, A., & Al-Lawati, A. (2010). On the development of a computer based diagnostic assessment tool to help in teaching and learning process. International Journal of Education and Development Using ICT, 6(1), 76–87. http://ijedict.dec.uwi.edu/viewissue.php?id=24

Alderson, J. C., & Huhta, A. (2005). The development of a suite of computer-based diagnostic tests based on the Common European Framework. Language Testing, 22(3), 301–320. https://doi.org/10.1191/0265532205lt310oa

Amanor-Mfoafo, N. K., Akrofi, O., Edonu, K. K., & Dowuona, E. N. (2020). Investigating the e-learning readiness of Ghanaian parents during COVID-19. European Journal of Education Studies, 7(10), 39-56. http://dx.doi.org/10.46827/ejes.v7i10.3275

Arcelay, I., Goti, A., Oyarbide-Zubillaga, A., Akyazi, T., Alberdi, E., & Garcia-Bringas, P. (2021). Definition of the future skills needs of job profiles in the renewable energy sector. Energies, 14(9), 2609. https://doi.org/10.3390/en14092609

Auxier, B., & Anderson, M. (2020, March 16). As schools close due to the coronavirus, some U.S. students face a digital 'homework gap.' Pew Research Center. https://www.pewresearch.org/short-reads/2020/03/16/as-schools-close-due-to-the-coronavirus-some-u-s-students-face-a-digital-homework-gap/

Bardini, C. (2015). Computer-based assessment of mathematics in PISA 2012. In Assessing mathematical literacy (pp. 173–188). Springer. https://doi.org/10.1007/978-3-319-10121-7_8

Benos, N., & Zotou, S. (2014). Education and economic growth: A meta-regression analysis. World Development, 64, 669–689. https://doi.org/10.1016/j.worlddev.2014.06.034

Berner, E. S., Webster, G. D., Shugerman, A. A., Jackson, J. R., Algina, J., Baker, A. L., Ball, E. v, Cobbs, C. G., Dennis, V. W., & Frenkel, E. P. (1994). Performance of four computer-based diagnostic systems. New England Journal of Medicine, 330(25), 1792–1796. https://doi.org/10.1056/NEJM199406233302506

Cha, S.-S., & Seo, B.-K. (2018). Smartphone use and smartphone addiction in middle school students in Korea: Prevalence, social networking service, and game use. Health Psychology Open, 5(1), 2055102918755046. https://doi.org/10.1177/2055102918755046

Chan, V. (2001). Readiness for learner autonomy: What do our learners tell us? Teaching in Higher Education, 6(4), 505–518. https://doi.org/10.1080/13562510120078045

Chorrojprasert, L. (2020). Learner readiness--why and how should they be ready?. LEARN Journal: Language Education and Acquisition Research Network, 13(1), 268–274.

Csapó, B., Ainley, J., Bennett, R. E., Latour, T., & Law, N. (2012). Technological issues for computer-based assessment. In Griffin, P., McGaw, B., & Care, E. (eds.), Assessment and teaching of 21st century skills, pp. 143–230. https://doi.org/10.1007/978-94-007-2324-5_4

Csapó, B., Molnár, G., & Nagy, J. (2014). Computer-based assessment of school readiness and early reasoning. Journal of Educational Psychology, 106(3), 639-650. https://doi.org/10.1037/a0035756

Dhiman, B. (2021). Impact of smartphone: A review on negative effects on students. Abid Ali, B.(2021). Impact of smartphone: A review on negative effects on students. PalArch's Journal of Archaeology of Egypt/Egyptology, 18(4), 5710–5718. http://dx.doi.org/10.2139/ssrn.4205892

Dolan, R., Hall, T. E., Banerjee, M., Chun, E., & Strangman, N. (2005). Applying principles of universal design to test delivery: The effect of computer-based read-aloud on test performance of high school students with learning disabilities. The Journal of Technology, Learning and Assessment, 3(7). https://ejournals.bc.edu/index.php/jtla/article/view/1660

Dray, B. J., Lowenthal, P. R., Miszkiewicz, M. J., Ruiz‐Primo, M. A., & Marczynski, K. (2011). Developing an instrument to assess student readiness for online learning: A validation study. Distance Education, 32(1), 29–47. https://doi.org/10.1080/01587919.2011.565496

Emine, C., & Kalelioglu, F. (2019). A fully online course experience from students' perspective: readiness, attitudes and thoughts. Turkish Online Journal of Distance Education, 20(3), 165–180. https://doi.org/10.17718/tojde.601934

Ferdiansyah, S. (2016). Moving from paper-based testing (PBT) to computer-based testing (CBT) for classroom use: Exploring the opportunities and challenges. In Proceeding of International Conference on Teacher Training and Education, 1(1), 514-519. https://jurnal.fkip.uns.ac.id/index.php/ictte/article/view/7659/5500

Fisher, W. P. (2007). Rating scale instrument quality criteria. Rasch Measurement Transactions, 21(1), 1095. https://www.rasch.org/rmt/rmt211.pdf

George-Jackson, C., & Gast, M. J. (2015). Addressing information gaps: Disparities in financial awareness and preparedness on the road to college. Journal of Student Financial Aid, 44(3), 3. https://doi.org/10.55504/0884-9153.1540

Greiff, S., Kretzschmar, A., Müller, J. C., Spinath, B., & Martin, R. (2014). The computer-based assessment of complex problem solving and how it is influenced by students' information and communication technology literacy. Journal of Educational Psychology, 106(3), 666-680. https://doi.org/10.1037/a0035426

Händel, M., Stephan, M., Gläser-Zikuda, M., Kopp, B., Bedenlier, S., & Ziegler, A. (2020). Digital readiness and its effects on higher education students' socio-emotional perceptions in the context of the COVID-19 pandemic. Journal of Research on Technology in Education, 54(2), 267-280. https://doi.org/10.1080/15391523.2020.1846147

Hanushek, E. A., & Woessmann, L. (2010). Education and economic growth. International Encyclopedia of Education, 2, pp. 245-252. https://hanushek.stanford.edu/publications/education-and-economic-growth

Infante-Moro, A., Infante-Moro, J.-C., & Gallardo-Pérez, J. (2019). The importance of ICTs for students as a competence for their future professional performance: The case of the faculty of business studies and tourism of the University of Huelva. Journal of New Approaches in Educational Research (NAER Journal), 8(2), 201–213. https://doi.org/10.7821/naer.2019.7.434

Irvin, R., & Macklin, A. S. (2007). Information and communication technology (ICT) literacy: Integration and assessment in higher education. Journal of Systemics, Cybernetics and Informatics, 5(4), 50–55. https://www.iiisci.org/Journal/pdv/sci/pdfs/P890541.pdf

Jamaluddin, A., Harjunowibowo, D., Rochim, A. M., Mahadmadi, F., Bulan, K. H., & Laksono, P. W. (2015). Implementation of RFID on computer based test (RF-CBT) system. In Proceedings of the Joint International Conference on Electric Vehicular Technology and Industrial, Mechanical, Electrical and Chemical Engineering (ICEVT & IMECE), 153–156. https://doi.org/10.1109/ICEVTIMECE.2015.7496645

Johan, J., & Supardi, I. (2015). Designing computer-based summative test for the eleventh grade students. Jurnal Pendidikan dan Pembelajaran Khatulistiwa, 4(10), 1-11. https://jurnal.untan.ac.id/index.php/jpdpb/article/view/11702

Kaarakainen, M.-T. (2019). ICT intentions and digital abilities of future labor market entrants in Finland. Nordic Journal of Working Life Studies, 9(2), 105-126. https://doi.org/10.18291/njwls.v9i2.114803

Kikusawa, M., Yamakawa, O. & Tanaka, T. (2006). The method and role of CBT in a classroom lecture of higher education. In T. Reeves & S. Yamashita (Eds.), Proceedings of E-Learn 2006--World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education (pp. 661-666). https://www.learntechlib.org/primary/p/23767/

Kumar, A., & Sherkhane, M. (2018). Assessment of gadgets addiction and its impact on health among undergraduates. International Journal of Community Medicine And Public Health, 5(8), 3624–3628. https://doi.org/10.18203/2394-6040.ijcmph20183109

Lestari, E., Stalmeijer, R. E., Widyandana, D., & Scherpbier, A. (2016). Understanding students' readiness for interprofessional learning in an Asian context: a mixed-methods study. BMC Medical Education, 16(1), 1–11. https://doi.org/10.1186/s12909-016-0704-3

Lim, E. C. H., Ong, B. K. C., Wilder-Smith, E. P. v, & Seet, R. C. S. (2006). Computer-based versus pen-and-paper testing: Students' perception. Annals-Academy of Medicine Singapore, 35(9), 599-603. https://scholarbank.nus.edu.sg/handle/10635/26849

Linacre, J. M. (2006). Data variance explained by Rasch measures. Rasch Measurement Transactions, 20(1), 1045. https://www.rasch.org/rmt/rmt201a.htm

Maqableh, M., Masa’deh, R. M. T., & Mohammed, A. B. (2015). The acceptance and use of computer based assessment in higher education. Journal of Software Engineering and Applications, 8(10), 557-574. https://doi.org/10.4236/jsea.2015.810053

Martin, F., Stamper, B., & Flowers, C. (2020). Examining student perception of readiness for online learning: Importance and confidence. Online Learning, 24(2), 38–58. https://doi.org/10.24059/olj.v24i2.2053

Martinussen, M., Torjussen, T., Storsve, O., & Hjerkinn, O. (2004). Pilot selection in the Norwegian Air Force: From paper and pencil to computer-based assessment. In Proceeding of the 40th Applied Military Psychology Symposium, pp. 1–7.

Massoud, S. L. (1991). Computer attitudes and computer knowledge of adult students. Journal of Educational Computing Research, 7(3), 269–291. https://doi.org/10.2190/HRRV-8EQV-U2TQ-C69G

Maulidiansyah, D., Meutia, I., & Istiyono, E. (2021). Computer-based two-tier diagnostic test to identify critical thinking skills in optical instrument. In The 6th International Seminar on Science Education (ISSE 2020), 413–418. https://doi.org/10.2991/assehr.k.210326.059

Nwoke, B. I., Osuji, C. U., & Agi, U. K. (2017). Influence of computer-based test (CBT) on examination malpractice in public examinations. IOSR Journal of Research & Method in Education, 7(2), 80–84.

Olsson, T., Samuelsson, U., & Viscovi, D. (2019). At risk of exclusion? Degrees of ICT access and literacy among senior citizens. Information, Communication & Society, 22(1), 55–72. https://doi.org/10.1080/1369118X.2017.1355007

Park, S., & Burford, S. (2013). A longitudinal study on the uses of mobile tablet devices and changes in digital media literacy of young adults. Educational Media International, 50(4), 266–280. https://doi.org/10.1080/09523987.2013.862365

Peat, M., & Franklin, S. (2002). Supporting student learning: The use of computer–based formative assessment modules. British Journal of Educational Technology, 33(5), 515–523. https://doi.org/10.1111/1467-8535.00288

Potosky, D., & Bobko, P. (1997). Computer versus paper-and-pencil administration mode and response distortion in noncognitive selection tests. Journal of Applied Psychology, 82(2), 293-299. https://doi.org/10.1037/0021-9010.82.2.293

Pramadya, W., & Indriati, D. (2019). Self-assessment profile on statistics using computer-based mathematical summative test. Journal of Physics: Conference Series, 1188(1), 012053. https://doi.org/10.1088/1742-6596/1188/1/012053

Pratama, M. O., Harinitha, D., Indriani, S., Denov, B., & Mahayana, D. (2020). Influence factors of social media and gadget addiction of adolescent in indonesia. Jurnal Sistem Informasi, 16(1), 16–24. https://jsi.cs.ui.ac.id/index.php/jsi/article/view/918

Rafique, G. M., Mahmood, K., Warraich, N. F., & Rehman, S. U. (2021). Readiness for online learning during COVID-19 pandemic: A survey of Pakistani LIS students. The Journal of Academic Librarianship, 47(3), 102346. https://doi.org/10.1016/j.acalib.2021.102346

Reyes, J. R. S., Grajo, J., Comia, L. N., Talento, M., Ebal, L. P. A., & Mendoza, J. J. (2021). Assessment of Filipino higher education students' readiness for e-learning during a pandemic: A Rasch Technique application. Philippine Journal of Science, 150(3), 1007–1018. https://doi.org/10.56899/150.03.34

Rudland, J. R., Schwartz, P., & Ali, A. (2011). Moving a formative test from a paper-based to a computer-based format. A student viewpoint. Medical Teacher, 33(9), 738–743. https://doi.org/10.3109/0142159X.2011.577119

Singh, M. K. K., & Samah, N. A. (2018). Impact of smartphone: A review on positive and negative effects on students. Asian Social Science, 14(11), 83–89. https://doi.org/10.5539/ass.v14n11p83

Stockwell, G. (2008). Investigating learner preparedness for and usage patterns of mobile learning. ReCALL, 20(3), 253–270. https://doi.org/10.1017/S0958344008000232

Syathroh, I. L., Musthafa, B., & Purnawarman, P. (2018). indonesian teachers' beliefs and experiences of computer-based English summative tests. International Journal of English Literature and Social Sciences, 3(6), 268319. https://doi.org/10.22161/ijels.3.6.36

Tang, Y. M., Chen, P. C., Law, K. M. Y., Wu, C.-H., Lau, Y., Guan, J., He, D., & Ho, G. T. S. (2021). Comparative analysis of student's live online learning readiness during the coronavirus (COVID-19) pandemic in the higher education sector. Computers & Education, 168, 104211. https://doi.org/10.1016/j.compedu.2021.104211

Tchamyou, V. S., Asongu, S. A., & Odhiambo, N. M. (2019). The role of ICT in modulating the effect of education and lifelong learning on income inequality and economic growth in Africa. African Development Review, 31(3), 261–274. https://doi.org/10.1111/1467-8268.12388

Terzis, V., & Economides, A. A. (2011). The acceptance and use of computer based assessment. Computers & Education, 56(4), 1032–1044. https://doi.org/10.1016/j.compedu.2010.11.017

Thurow, L. C. (1972). Education and economic equality. The Public Interest, 28, 66-81. https://www.nationalaffairs.com/public_interest/detail/education-and-economic-equality

Tomasik, M. J., Berger, S., & Moser, U. (2018). On the development of a computer-based tool for formative student assessment: Epistemological, methodological, and practical issues. Frontiers in Psychology, 9, 2245. https://doi.org/10.3389/fpsyg.2018.02245

Ukwueze, C. A., & Uzoagba, O. N. (2021). ICT literacy and readiness for computer based test among public secondary school students in Anambra State. New Media and Mass Communication, 97, 1-14. https://www.iiste.org/Journals/index.php/NMMC/article/view/57119

van Groen, M. M., & Eggen, T. J. H. M. (2019). Educational test approaches: The suitability of computer-based test types for assessment and evaluation in formative and summative contexts. Journal of Applied Testing Technology, 21(1), 12–24. https://www.jattjournal.net/index.php/atp/article/view/146484

van Lent, G., & Global, E. T. S. (2009). Risks and benefits of CBT versus PBT in high-stakes testing. The Transition to Computer-Based Assessment, 83.

van Rooij, S. W., & Zirkle, K. (2016). Balancing pedagogy, student readiness and accessibility: A case study in collaborative online course development. The Internet and Higher Education, 28, 1–7. https://doi.org/10.1016/j.iheduc.2015.08.001

Wardhani, F. P. (2018). Student gadget addiction behavior in the perspective of respectful framework. Konselor, 7(3), 116–123. https://doi.org/10.24036/0201872100184-0-00

Widodo, A., & Wartoyo, F. X. (2020). Lockdown and gadget addicted phenomenon: Changes in social behavior of school age children during the Covid-19 pandemic in Mataram City. In Proceedings of the 4th International Conference on Learning Innovation and Quality Education, 1–8. https://doi.org/10.1145/3452144.3452163

Wiechmann, D., & Ryan, A. M. (2003). Reactions to computerised testing in selection contexts. International Journal of Selection and Assessment, 11(2‐3), 215–229. https://doi.org/10.1111/1468-2389.00245

Yamamoto, K., Shin, H. J., & Khorramdel, L. (2019). Introduction of multistage adaptive testing design in PISA 2018. In OECD Education Working Papers No. 209. https://dx.doi.org/10.1787/b9435d4b-en

Yu, W., & Iwashita, N. (2021). Comparison of test performance on paper-based testing (PBT) and computer-based testing (CBT) by English-majored undergraduate students in China. Language Testing in Asia, 11(1), 1–21. https://doi.org/10.1186/s40468-021-00147-0

Zakwandi, R. (2022). Development of computational thinking and scientific literacy assessment instruments for students in physics subjects with computerized adaptive test [Master Thesis]. Universitas Negeri Yogyakarta, Yogyakarta.

DOI: https://doi.org/10.21831/pep.v27i1.56005