Empirical lecturers’ and students’ satisfaction assessment in e-learning systems based on the usage metrics

Sulis Sandiwarno

doi:10.21831/reid.v7i2.39642

Empirical lecturers’ and students’ satisfaction assessment in e-learning systems based on the usage metrics

Sulis Sandiwarno, Universitas Mercu Buana, Indonesia

10.21831/reid.v7i2.39642

Abstract

Nowadays, in the pandemic of COVID-19, e-learning systems have been widely used to facilitate teaching and learning processes between lecturers and students. Assessing lecturers’ and students’ satisfaction with e-learning systems has become essential in improving the quality of education for higher learning institutions. Most existing approaches have attempted to assess users’ satisfaction based on System Usability Scale (SUS). On the other hand, different studies proposed usage-based metrics (completion rate, task duration, and mouse or cursor distance) which assess users’ satisfaction based on how they use and interact with the system. However, the cursor or mouse distance metric does not consider the effectiveness of navigation in e-learning systems, and such approaches measure either lecturers’ or students’ satisfaction independently. Towards this end, we propose a lostness metric to replace the click or cursor distance metric for assessing lecturers’ and students’ satisfaction with using e-learning systems. Furthermore, to obtain a deep analysis of users’ satisfaction, we tandem the usage-based metric (i.e., completion rate, task duration, and lostness) and the SUS metric. The evaluation results indicate that the proposed approach can precisely predict users’ satisfaction with e-learning systems.

Keywords

e-learning; satisfaction; usage-based metrics; SUS

Full Text:

PDF

References

Ahn, J., Kim, K., & Proctor, R. W. (2018). Comparison of mobile web browsers for smartphones. Journal of Computer Information Systems, 58(1), 10-18. https://doi.org/10.1080/08874417.2016.1180652

AlGhannam, B. A., Albustan, S. A., Al-Hassan, A. A., & Albustan, L. A. (2017). Towards a standard Arabic system usability scale: Psychometric evaluation using communication disorder app. International Journal of Human–Computer Interaction, 34(9), 1–6. https://doi.org/10.1080/10447318.2017.1388099

Almarashdeh, I. (2016). Sharing instructors experience of learning management system: A technology perspective of user satisfaction in distance learning course. Computers in Human Behavior, 63, 249–255. https://doi.org/10.1016/j.chb.2016.05.013

Asoodar, M., Vaezi, S., & Izanloo, B. (2016a). Framework to improve e-learner satisfaction and further strengthen e-learning implementation. Computers in Human Behavior, 63, 704–716. https://doi.org/10.1016/j.chb.2016.05.060

Asoodar, M., Vaezi, S., & Izanloo, B. (2016b). Framework to improve e-learner satisfaction and further strengthen e-learning implementation. Computers in Human Behavior, 63, 704–716. https://doi.org/10.1016/j.chb.2016.05.060

Berkman, M. İ., Karahoca, D., & Karahoca, A. (2018). A measurement and structural model for usability evaluation of shared workspace groupware. International Journal of Human-Computer Interaction, 34(1), 35–56. https://doi.org/10.1080/10447318.2017.1326578

Borkowska, A., & Jach, K. (2017). Pre-testing of Polish translation of System Usability Scale (SUS). Advances in Intelligent Systems and Computing, 521, 143-153. https://doi.org/10.1007/978-3-319-46583-8_12

Bringula, R. P. (2013). Influence of faculty- and web portal design-related factors on web portal usability: A hierarchical regression analysis. Computers and Education, 68, 187-198. https://doi.org/10.1016/j.compedu.2013.05.008

Caputi, V., & Garrido, A. (2015). Student-oriented planning of e-learning contents for Moodle. Journal of Network and Computer Applications, 53, 115–127. https://doi.org/10.1016/j.jnca.2015.04.001

Casamayor, A., Amandi, A., & Campo, M. (2009). Intelligent assistance for teachers in collaborative e-learning environments. Computers and Education, 53(4), 1147–1154. https://doi.org/10.1016/j.compedu.2009.05.025

Chen, P.-H., & Adesope, O. (2016). The effects of need satisfaction on EFL online learner satisfaction. Distance Education, 37(1), 89–106. https://doi.org/10.1080/01587919.2016.1155962

Cohen, A., & Baruth, O. (2017). Personality, learning, and satisfaction in fully online academic courses. Computers in Human Behavior, 72, 1–12. https://doi.org/10.1016/j.chb.2017.02.030

Curcio, K., Santana, R., Reinehr, S., & Malucelli, A. (2019). Usability in agile software development: A tertiary study. Computer Standards and Interfaces, 64, 61-77. https://doi.org/10.1016/j.csi.2018.12.003

Gameel, B. G. (2017). Learner satisfaction with massive open online courses. American Journal of Distance Education, 31(2), 98–111. https://doi.org/10.1080/08923647.2017.1300462

Haron, H., Aziz, N. H. N., & Harun, A. (2017). A conceptual model participatory engagement within e-learning community. Procedia Computer Science, 116, 242–250. https://doi.org/10.1016/j.procs.2017.10.046

Harrati, N., Bouchrika, I., Tari, A., & Ladjailia, A. (2016). Exploring user satisfaction for e-learning systems via usage-based metrics and system usability scale analysis. Computers in Human Behavior, 61, 463–471. https://doi.org/10.1016/j.chb.2016.03.051

Hong, J. C., Tai, K. H., Hwang, M. Y., Kuo, Y. C., & Chen, J. S. (2017). Internet cognitive failure relevant to users’ satisfaction with content and interface design to reflect continuance intention to use a government e-learning system. Computers in Human Behavior, 66, 353–362. https://doi.org/10.1016/j.chb.2016.08.044

Horvat, A., Dobrota, M., Krsmanovic, M., & Cudanov, M. (2015). Student perception of Moodle learning management system: A satisfaction and significance analysis. Interactive Learning Environments, 23(4), 515–527. https://doi.org/10.1080/10494820.2013.788033

Ifinedo, P., Pyke, J., & Anwar, A. (2018). Business undergraduates’ perceived use outcomes of Moodle in a blended learning environment: The roles of usability factors and external support. Telematics and Informatics, 35(1), 93–102). https://doi.org/10.1016/j.tele.2017.10.001

Kerimbayev, N., Kultan, J., Abdykarimova, S., & Akramova, A. (2017). LMS Moodle: Distance international education in cooperation of higher education institutions of different countries. Education and Information Technologies, 22(5), 2125–2139. https://doi.org/10.1007/s10639-016-9534-5

Kim, J. (2013). Influence of group size on students’ participation in online discussion forums. Computers & Education, 62, 123-129. https://doi.org/10.1016/j.compedu.2012.10.025

Koohang, A., Paliszkiewicz, J., Gołuchowski, J., & Nord, J. H. (2016). Active learning for knowledge construction in e-learning: A replication study. Journal of Computer Information Systems, 56(3), 238–243. https://doi.org/10.1080/08874417.2016.1153914

Ku, H. Y., Tseng, H. W., & Akarasriworn, C. (2013). Collaboration factors, teamwork satisfaction, and student attitudes toward online collaborative learning. Computers in Human Behavior, 29(3), 922–929. https://doi.org/10.1016/j.chb.2012.12.019

Liberona, D., & Fuenzalida, D. (2014). Use of Moodle platforms in higher education: A Chilean case. Communications in Computer and Information Science, 446, 124–134. https://doi.org/10.1007/978-3-319-10671-7_12

Lin, J. W. (2018). Effects of an online team project-based learning environment with group awareness and peer evaluation on socially shared regulation of learning and self-regulated learning. Behaviour and Information Technology, 37(5), 445–461. https://doi.org/10.1080/0144929X.2018.1451558

Mentes, A., & Turan, A. H. (2012). Assessing the usability of university websites: An empirical study on Namik Kemal University. Turkish Online Journal of Educational Technology, 11(3), 61-69. http://www.tojet.net/articles/v11i3/1136.pdf

Muñoz, A., Delgado, R., Rubio, E., Grilo, C., & Basto-Fernandes, V. (2017). Forum participation plugin for Moodle: Development and discussion. Procedia Computer Science, 121, 982–989. https://doi.org/10.1016/j.procs.2017.11.127

Navimipour, N. J., & Zareie, B. (2015). A model for assessing the impact of e-learning systems on employees’ satisfaction. Computers in Human Behavior, 53, 475–485. https://doi.org/10.1016/j.chb.2015.07.026

Sadikin, M. (2017). Mining relation extraction based on pattern learning approach. Indonesian Journal of Electrical Engineering and Computer Science, 6(1), 50-57. https://doi.org/10.11591/ijeecs.v6.i1.pp50-57

Sadikin, M., Fanany, M. I., & Basaruddin, T. (2016). A new data representation based on training data characteristics to extract drug name entity in medical text. Computational Intelligence and Neuroscience, 3483528. https://doi.org/10.1155/2016/3483528

Sandiwarno, S. (2016). Perancangan model e-learning berbasis collaborative video conference learning guna mendapatkan hasil pembelajaran yang efektif dan efisien. Jurnal Ilmiah FIFO, 8(2), 191-200. https://doi.org/10.22441/fifo.v8i2.1314

Smith, P. A. (1996). Towards a practical measure of hypertext usability. Interacting with Computers, 8(4), 365–381. https://doi.org/10.1016/S0953-5438(97)83779-4

Sun, J. (2016). Multi-dimensional alignment between online instruction and course technology: A learner-centered perspective. Computers and Education, 101, 102–114. https://doi.org/10.1016/j.compedu.2016.06.003

Tullis, T., & Albert, B. (2013). Measuring the user experience: Collecting, analyzing, and presenting usability metrics (2nd ed.). Elsevier. https://doi.org/10.1016/C2011-0-00016-9

Wagner, N., Hassanein, K., & Head, M. (2014). The impact of age on website usability. Computers in Human Behavior, 37, 270-282. https://doi.org/10.1016/j.chb.2014.05.003

Zhang, S., Liu, Q., Chen, W., Wang, Q., & Huang, Z. (2017). Interactive networks and social knowledge construction behavioral patterns in primary school teachers’ online collaborative learning activities. Computers and Education, 104, 1–17. https://doi.org/10.1016/j.compedu.2016.10.011

DOI: https://doi.org/10.21831/reid.v7i2.39642