A Statistical Analysis of System Usability Scale (SUS) Evaluations in Online Learning Platform
Abstract
This study evaluates the usability of a Moodle-based online learning system at a university using the System Usability Scale (SUS). Introduced in 2019, the platform has been instrumental in facilitating access to educational resources, enhancing the distribution of course materials, and supporting academic activities through interactive features. The SUS was employed to gather subjective feedback from 120 student respondents, providing insights into the system's efficiency, effectiveness, and user satisfaction. Statistical analysis was performed using methods including the Shapiro-Wilk test to address the skewed distribution of the original dataset with a mean SUS score of 71.52 and a high standard deviation of 16.18, indicating varied user experiences. Analysis of sample means was also conducted to achieve a more stable estimate of usability, resulting in a mean of 71.51 with a significantly reduced standard deviation of 2.44, suggesting a more consistent assessment across users. The final SUS score placed the system in the 'C+' category according to the Sauro and Lewis grading scale, indicating that while usability is acceptable, substantial improvements are necessary.
Downloads
References
U. Mukarromah and W. Wijayanti, “Implementation of the online learning at vocational high school during Covid-19: Between obligations and barriers,” Jurnal Pendidikan Vokasi, vol. 11, no. 1, Mar. 2021, doi: 10.21831/jpv.v11i1.37110.
V. Singh and A. Thurman, “How Many Ways Can We Define Online Learning? A Systematic Literature Review of Definitions of Online Learning (1988-2018),” American Journal of Distance Education, vol. 33, no. 4, pp. 289–306, Oct. 2019, doi: 10.1080/08923647.2019.1663082.
E. Enoh, B. Qamaruzzaman, and Q. Y. Zaqiyah, “Pengembangan dan Implementasi E-Learning di Universitas Islam Bandung,” Ta’dib Jurnal Pendidikan Islam, vol. 12, no. 2, pp. 573–584, Aug. 2023, doi: 10.29313/TJPI.V12I2.12312.
H. C. Kuong, “Enhancing Online Learning Experience: From Learners’ Perspective,” Procedia Soc Behav Sci, vol. 191, pp. 1002–1005, Jun. 2015, doi: 10.1016/j.sbspro.2015.04.403.
N. K. A. Suwastini, K. I. R. Dewi, and G. R. Dantes, “Benefits of online learning according to recent studies,” Jurnal Inovasi dan Teknologi Pembelajaran, vol. 10, no. 1, pp. 32–42, Mar. 2023, doi: 10.17977/UM031V10I12023P032.
L. Pei and H. Wu, “Does online learning work better than offline learning in undergraduate medical education? A systematic review and meta-analysis,” Med Educ Online, vol. 24, no. 1, p. 1666538, Jan. 2019, doi: 10.1080/10872981.2019.1666538.
P. Paudel, “Online Education: Benefits, Challenges and Strategies During and After COVID-19 in Higher Education,” International Journal on Studies in Education, vol. 3, no. 2, pp. 70–85, Sep. 2021, doi: 10.46328/IJONSE.32.
M. D. B. Castro and G. M. Tumibay, “A literature review: efficacy of online learning courses for higher education institution using meta-analysis,” Educ Inf Technol (Dordr), vol. 26, no. 2, pp. 1367–1385, Mar. 2021, doi: 10.1007/S10639-019-10027-Z/METRICS.
H. N. Y. Nhi and T. T. Nhan, “Online Education During The Covid-19 Pandemic - Opportunities And Challenges: A Case Study In The South Of Vietnam,” European Journal of Open Education and E-learning Studies, vol. 7, no. 1, Apr. 2022, doi: 10.46827/ejoe.v7i1.4222.
A. Kilburn, B. Kilburn, and T. Cates, “Drivers of Student Retention: System Availability, Privacy, Value And Loyalty In Online Higher Education,” Academy of Educational Leadership Journal, vol. 18, no. 4, 2014.
W. Bhuasiri, O. Xaymoungkhoun, H. Zo, J. J. Rho, and A. P. Ciganek, “Critical success factors for e-learning in developing countries: A comparative analysis between ICT experts and faculty,” Comput Educ, vol. 58, no. 2, pp. 843–855, Feb. 2012, doi: 10.1016/J.COMPEDU.2011.10.010.
J. Brooke, “SUS: a retrospective,” J Usability Stud, vol. 8, no. 2, pp. 29–40, 2013.
B. Klug, “An Overview of the System Usability Scale in Library Website and System Usability Testing,” Weave: Journal of Library User Experience, vol. 1, no. 6, Apr. 2017, doi: 10.3998/weave.12535642.0001.602.
J. Brooke, “SUS-a quick and dirty usability scale,” in Usability Evaluation In Industry, P. W. Jordan, B. Thomas, and I. Lyall, Eds., London, Bristol: Taylor & Francis, 1996, pp. 189–194.
S. Manikandan, “Measures of central tendency: The mean,” J Pharmacol Pharmacother, vol. 2, no. 2, pp. 140–142, Jun. 2011, doi: 10.4103/0976-500X.81920.
M. A. Barri, “A Simulation Showing the Role of Central Limit Theorem in Handling Non-Normal Distributions,” Am J Educ Res, vol. 7, no. 8, pp. 591–598, Aug. 2019, doi: 10.12691/education-7-8-8.
N. Clark, M. Dabkowski, P. J. Driscoll, D. Kennedy, I. Kloo, and H. Shi, “Empirical Decision Rules for Improving the Uncertainty Reporting of Small Sample System Usability Scale Scores,” Int J Hum Comput Interact, vol. 37, no. 13, pp. 1191–1206, Aug. 2021, doi: 10.1080/10447318.2020.1870831.
P. Vlachogianni and N. Tselios, “Perceived usability evaluation of educational technology using the System Usability Scale (SUS): A systematic review,” Journal of Research on Technology in Education, vol. 54, no. 3, pp. 392–409, May 2022, doi: 10.1080/15391523.2020.1867938.
F. G. Sembodo, G. F. Fitriana, and N. A. Prasetyo, “Evaluasi Usability Website Shopee Menggunakan System Usability Scale (SUS),” 2021.
K. Sagar and A. Saha, “Quantitative usability assessment relying on experiential and specific task-based SUS ratings,” Journal of Statistics and Management Systems, vol. 23, no. 2, pp. 333–348, Feb. 2020, doi: 10.1080/09720510.2020.1736316.
J. Sauro and J. R. Lewis, Quantifying the User Experience: Practical Statistics for User Research, 2nd Edition. Cambridge: Elsevier, 2016. doi: 10.1016/C2010-0-65192-3.
J. P. Royston, “An Extension of Shapiro and Wilk’s W Test for Normality to Large Samples,” J R Stat Soc Ser C Appl Stat, vol. 31, no. 2, pp. 115–124, Jun. 1982, doi: 10.2307/2347973.
S. G. Kwak and J. H. Kim, “Introduction Basic Concepts of Central Limit Theorem Central limit theorem: the cornerstone of modern statistics KJA,” Korean J Anesthesiol, vol. 70, no. 2, pp. 144–156, 2017, doi: 10.4097/kjae.2017.70.2.144.
R. Bono, M. J. Blanca, J. Arnau, and J. Gómez-Benito, “Non-normal distributions commonly used in health, education, and social sciences: A systematic review,” Front Psychol, vol. 8, no. SEP, p. 267072, Sep. 2017, doi: 10.3389/FPSYG.2017.01602/BIBTEX.
Abstract views: 544 times
Download PDF: 299 times
Copyright (c) 2024 Journal of Information Systems and Informatics
This work is licensed under a Creative Commons Attribution 4.0 International License.
- I certify that I have read, understand and agreed to the Journal of Information Systems and Informatics (Journal-ISI) submission guidelines, policies and submission declaration. Submission already using the provided template.
- I certify that all authors have approved the publication of this and there is no conflict of interest.
- I confirm that the manuscript is the authors' original work and the manuscript has not received prior publication and is not under consideration for publication elsewhere and has not been previously published.
- I confirm that all authors listed on the title page have contributed significantly to the work, have read the manuscript, attest to the validity and legitimacy of the data and its interpretation, and agree to its submission.
- I confirm that the paper now submitted is not copied or plagiarized version of some other published work.
- I declare that I shall not submit the paper for publication in any other Journal or Magazine till the decision is made by journal editors.
- If the paper is finally accepted by the journal for publication, I confirm that I will either publish the paper immediately or withdraw it according to withdrawal policies
- I Agree that the paper published by this journal, I transfer copyright or assign exclusive rights to the publisher (including commercial rights)
_1.png){kind=logo}
