A Statistical Analysis of System Usability Scale (SUS) Evaluations in Online Learning Platform

  • Ozzi Suria Universitas Mercu Buana Yogyakarta, Indonesia
Keywords: online learning, System Usability Scale (SUS), Moodle, statistical analysis, usability improvement

Abstract

This study evaluates the usability of a Moodle-based online learning system at a university using the System Usability Scale (SUS). Introduced in 2019, the platform has been instrumental in facilitating access to educational resources, enhancing the distribution of course materials, and supporting academic activities through interactive features. The SUS was employed to gather subjective feedback from 120 student respondents, providing insights into the system's efficiency, effectiveness, and user satisfaction. Statistical analysis was performed using methods including the Shapiro-Wilk test to address the skewed distribution of the original dataset with a mean SUS score of 71.52 and a high standard deviation of 16.18, indicating varied user experiences. Analysis of sample means was also conducted to achieve a more stable estimate of usability, resulting in a mean of 71.51 with a significantly reduced standard deviation of 2.44, suggesting a more consistent assessment across users. The final SUS score placed the system in the 'C+' category according to the Sauro and Lewis grading scale, indicating that while usability is acceptable, substantial improvements are necessary.

Downloads

Download data is not yet available.

References

U. Mukarromah and W. Wijayanti, “Implementation of the online learning at vocational high school during Covid-19: Between obligations and barriers,” Jurnal Pendidikan Vokasi, vol. 11, no. 1, Mar. 2021, doi: 10.21831/jpv.v11i1.37110.

V. Singh and A. Thurman, “How Many Ways Can We Define Online Learning? A Systematic Literature Review of Definitions of Online Learning (1988-2018),” American Journal of Distance Education, vol. 33, no. 4, pp. 289–306, Oct. 2019, doi: 10.1080/08923647.2019.1663082.

E. Enoh, B. Qamaruzzaman, and Q. Y. Zaqiyah, “Pengembangan dan Implementasi E-Learning di Universitas Islam Bandung,” Ta’dib Jurnal Pendidikan Islam, vol. 12, no. 2, pp. 573–584, Aug. 2023, doi: 10.29313/TJPI.V12I2.12312.

H. C. Kuong, “Enhancing Online Learning Experience: From Learners’ Perspective,” Procedia Soc Behav Sci, vol. 191, pp. 1002–1005, Jun. 2015, doi: 10.1016/j.sbspro.2015.04.403.

N. K. A. Suwastini, K. I. R. Dewi, and G. R. Dantes, “Benefits of online learning according to recent studies,” Jurnal Inovasi dan Teknologi Pembelajaran, vol. 10, no. 1, pp. 32–42, Mar. 2023, doi: 10.17977/UM031V10I12023P032.

L. Pei and H. Wu, “Does online learning work better than offline learning in undergraduate medical education? A systematic review and meta-analysis,” Med Educ Online, vol. 24, no. 1, p. 1666538, Jan. 2019, doi: 10.1080/10872981.2019.1666538.

P. Paudel, “Online Education: Benefits, Challenges and Strategies During and After COVID-19 in Higher Education,” International Journal on Studies in Education, vol. 3, no. 2, pp. 70–85, Sep. 2021, doi: 10.46328/IJONSE.32.

M. D. B. Castro and G. M. Tumibay, “A literature review: efficacy of online learning courses for higher education institution using meta-analysis,” Educ Inf Technol (Dordr), vol. 26, no. 2, pp. 1367–1385, Mar. 2021, doi: 10.1007/S10639-019-10027-Z/METRICS.

H. N. Y. Nhi and T. T. Nhan, “Online Education During The Covid-19 Pandemic - Opportunities And Challenges: A Case Study In The South Of Vietnam,” European Journal of Open Education and E-learning Studies, vol. 7, no. 1, Apr. 2022, doi: 10.46827/ejoe.v7i1.4222.

A. Kilburn, B. Kilburn, and T. Cates, “Drivers of Student Retention: System Availability, Privacy, Value And Loyalty In Online Higher Education,” Academy of Educational Leadership Journal, vol. 18, no. 4, 2014.

W. Bhuasiri, O. Xaymoungkhoun, H. Zo, J. J. Rho, and A. P. Ciganek, “Critical success factors for e-learning in developing countries: A comparative analysis between ICT experts and faculty,” Comput Educ, vol. 58, no. 2, pp. 843–855, Feb. 2012, doi: 10.1016/J.COMPEDU.2011.10.010.

J. Brooke, “SUS: a retrospective,” J Usability Stud, vol. 8, no. 2, pp. 29–40, 2013.

B. Klug, “An Overview of the System Usability Scale in Library Website and System Usability Testing,” Weave: Journal of Library User Experience, vol. 1, no. 6, Apr. 2017, doi: 10.3998/weave.12535642.0001.602.

J. Brooke, “SUS-a quick and dirty usability scale,” in Usability Evaluation In Industry, P. W. Jordan, B. Thomas, and I. Lyall, Eds., London, Bristol: Taylor & Francis, 1996, pp. 189–194.

S. Manikandan, “Measures of central tendency: The mean,” J Pharmacol Pharmacother, vol. 2, no. 2, pp. 140–142, Jun. 2011, doi: 10.4103/0976-500X.81920.

M. A. Barri, “A Simulation Showing the Role of Central Limit Theorem in Handling Non-Normal Distributions,” Am J Educ Res, vol. 7, no. 8, pp. 591–598, Aug. 2019, doi: 10.12691/education-7-8-8.

N. Clark, M. Dabkowski, P. J. Driscoll, D. Kennedy, I. Kloo, and H. Shi, “Empirical Decision Rules for Improving the Uncertainty Reporting of Small Sample System Usability Scale Scores,” Int J Hum Comput Interact, vol. 37, no. 13, pp. 1191–1206, Aug. 2021, doi: 10.1080/10447318.2020.1870831.

P. Vlachogianni and N. Tselios, “Perceived usability evaluation of educational technology using the System Usability Scale (SUS): A systematic review,” Journal of Research on Technology in Education, vol. 54, no. 3, pp. 392–409, May 2022, doi: 10.1080/15391523.2020.1867938.

F. G. Sembodo, G. F. Fitriana, and N. A. Prasetyo, “Evaluasi Usability Website Shopee Menggunakan System Usability Scale (SUS),” 2021.

K. Sagar and A. Saha, “Quantitative usability assessment relying on experiential and specific task-based SUS ratings,” Journal of Statistics and Management Systems, vol. 23, no. 2, pp. 333–348, Feb. 2020, doi: 10.1080/09720510.2020.1736316.

J. Sauro and J. R. Lewis, Quantifying the User Experience: Practical Statistics for User Research, 2nd Edition. Cambridge: Elsevier, 2016. doi: 10.1016/C2010-0-65192-3.

J. P. Royston, “An Extension of Shapiro and Wilk’s W Test for Normality to Large Samples,” J R Stat Soc Ser C Appl Stat, vol. 31, no. 2, pp. 115–124, Jun. 1982, doi: 10.2307/2347973.

S. G. Kwak and J. H. Kim, “Introduction Basic Concepts of Central Limit Theorem Central limit theorem: the cornerstone of modern statistics KJA,” Korean J Anesthesiol, vol. 70, no. 2, pp. 144–156, 2017, doi: 10.4097/kjae.2017.70.2.144.

R. Bono, M. J. Blanca, J. Arnau, and J. Gómez-Benito, “Non-normal distributions commonly used in health, education, and social sciences: A systematic review,” Front Psychol, vol. 8, no. SEP, p. 267072, Sep. 2017, doi: 10.3389/FPSYG.2017.01602/BIBTEX.

Published
2024-06-13
Abstract views: 544 times
Download PDF: 299 times
How to Cite
Suria, O. (2024). A Statistical Analysis of System Usability Scale (SUS) Evaluations in Online Learning Platform. Journal of Information Systems and Informatics, 6(2), 992-1007. https://doi.org/10.51519/journalisi.v6i2.750