Analysis of User Experience on SPOTA UNTAN Using Heuristic Walkthrough Method

SPOTA UNTAN is an abbreviation for Sistem Pendukung Outline Tugas Akhir for the Informatics study program at Tanjungpura University. SPOTA UNTAN was developed within the Informatics study program at Tanjungpura University in 2008. SPOTA UNTAN is an information system that support students in submitting their final project topics. As for now, SPOTA UNTAN has undergone several updates, including the new features to enhance the information system functionality. As its interface has undergone some changes, such as the addition of content, however, the CSS framework (colors, fonts, shapes, etc.) still retains the original design and not yet updated to align with the user experience. Therefore, several analyses are needed to address these uncertainties from a user interface perspective by usability tests. The study will apply the Heuristic Walkthrough method, which combines Cognitive Walkthrough and Heuristic Evaluation. The analysis will involve 5 expert evaluators in UI/UX. This study result showed a total of 39 recommendations for problem resolution, categorized into functional feature improvements and interface enhancements.


INTRODUCTION
The rapid advancement of technology in the 21st century has made it imperative for educational institutions to integrate digital systems into their infrastructures.Recognizing the need to keep pace with this evolution, the Informatics Department at Tanjungpura University has introduced the Sistem Pendukung Outline Tugas Akhir (SPOTA).This web-based information system is designed to simplify the administrative tasks associated with final projects.SPOTA UNTAN functions as a centralized hub for Informatics students, offering a comprehensive suite of features such as consultations with lecturers, outline submissions, and realtime updates on upcoming seminars and deadlines.Despite its functional improvements over time, SPOTA UNTAN's user interface has remained somewhat static, particularly with regard to its CSS framework.The interface is often the first point of contact between the students and the system, making its design and usability paramount for fostering a positive user experience.The lack of interface updates is a critical issue that not only detracts from the system's visual appeal but also could compromise its overall effectiveness and ease of use.
The primary objective of this research is to conduct an in-depth evaluation of SPOTA UNTAN's user interface with the aim of enhancing user experience.We intend to perform a rigorous analysis to identify any usability issues and present specific, actionable recommendations for improvements.This will help ensure that the system is more closely aligned with the needs and expectations of its primary stakeholders-students and faculty members alike.
To fulfill our research goals, we will employ the heuristic walkthrough method, a hybrid approach that amalgamates the advantages of both heuristic evaluation and cognitive walkthrough techniques [1].The heuristic evaluation phase will involve industry experts meticulously examining the interface to pinpoint usability flaws [2].Meanwhile, the cognitive walkthrough will simulate real-world user interactions to identify any roadblocks or areas of confusion.The integration of these methods allows for a comprehensive assessment, providing a multi-faceted view of the interface's strengths, weaknesses, and areas for improvement.
The implications of this research could be groundbreaking for the effectiveness of SPOTA UNTAN as a comprehensive support system for final projects in the Informatics program.The recommendations generated from this study promise to elevate the user experience to new heights and enhance overall administrative efficiency.Such improvements are likely to result in a win-win situation, benefiting both students and faculty members of Tanjungpura University, while also setting a precedent for similar initiatives in other educational settings.

METHODS
This research employs the Heuristic Walkthrough method, which is structured into two distinct phases: the Preparation Phase and the Execution Phase, as illustrated in Figure 1.The specifics of each phase are outlined below.

Literature Study
The literature review conducted in this research includes journals, books, scientific papers, websites, and similar studies that have been previously conducted.The findings from the literature review serve as references for conducting analysis and measurements in the research.

Task Scenario Determination
The selection of task scenarios in Cognitive Walkthrough is based on the most important tasks on system, in other meaning the system which was the core activities that are directly related to the user and tasks that have significant consequences if errors occur [3].Every task on the SPOTA UNTAN page is provided with a task completion scenario that consists of the tasks to be completed (based on the most frequently accessed pages), the steps to complete the tasks, and the input data.Based on the data above (underlined), there are 6 (six) query strings from the student dashboard with the highest visit data.Therefore, the following tasks will be used for cognitive walkthrough evaluation.

Evaluator Selection
The criteria of evaluators which will serve as inspectors in this study are individuals who are not end-users of SPOTA UNTAN and individual who have a deep understanding and knowledge of digital product development as well as a background related to UI/UX design, computer science, etc.The analysis will involve 5 evaluators.Caroline, Helen Sastypratiwi, at all | 1145

Guidance for Evaluator
The guide to the evaluators will be delivered both verbally and in written form through online platforms.As a documentation medium for the evaluators, the guidance, along with the table for collecting the analysis results, will be compiled in a document and shared through Google Docs so evaluators able to observe and analyze it.The following instruction for the evaluator regarding each analysis step that will be conducted: 1. Task Scenario: The evaluator will be provided with individual dummy accounts to analyze SPOTA UNTAN by performing the task scenarios attached to the form.The evaluator can only complete the task scenarios once without repetition, and the process will be screen recorded.2. Thought-Provoking Questions: The evaluator will answer thoughtprovoking questions [4].Not all thought-provoking questions need to be answered by the evaluator, only the ones related to problematic issues.3. Heuristic Evaluation: Next, the evaluator will explore SPOTA UNTAN and identify any issues encountered.Subsequently, the evaluator will provide suggestions, determine Heuristic Numbers, and assign Severity Ratings to each identified problem.

Retrieving Cognitive Walkthrough Analysis Result
In this phase, the evaluators will analyze SPOTA based on the task scenarios that are shared in the form and then record the issues they encounter during the task scenario execution in the table on the form based on thought provoking question list.The task scenario will be completed from start to finish in one continuous run, and the entire process will be documented using screen recording.After completing the task scenarios, the evaluator will fill out a table listing the issues encountered based on the thought-provoking questions.Only the thoughtprovoking questions that reveal problems will be filled out in the table .After obtaining all the problem findings from each evaluator, the author will proceed to map the issues.This step is taken if there are problems found by multiple evaluators that are similar or the same.
Table 1.Thought Provoking Question [4] No Thought Provoking Question 1 Does the effect of that action align with the user's intention at that moment? 2 Will the user perceive that the control to perform that action is available?3 Can users easily find the control? 4 After users find the correct control, will they know that it is what they need?5 After the action taken, will the users understand the feedback they receive?

Retrieving Heuristic Evaluation Analysis Result
In general, in this phase the evaluators will conduct a comprehensive exploration of SPOTA while writing down the identified issues on the table of document.The collection of heuristic evaluation analysis results is divided into two stages: 1. Stage One: During this stage, while revisiting SPOTA, each evaluator will fill in a table that includes columns for problem details, improvement suggestions for the problems, classifying the problems based on heuristic numbers, and determining the severity ratings for each problem.After obtaining all the problem findings from each evaluator, the author will proceed to map the issues.This step is taken if there are problems found by multiple evaluators that are similar or the same.2. Stage Two: In this stage, the previously mapped heuristic evaluation problem findings will be distributed to each evaluator again for reclassifying their heuristic numbers and severity ratings.This is done to standardize the heuristic number values for some problems that have been identified by more than one evaluator.

Organize Improvement Recommendation
Once the issues have been mapped, the next step is to organize the editorial recommendations regarding the aspects of the interface that require to be developed.The recommendations based on the prior identified issues and the suggestions provided by the evaluators in the document.

Problem Mapping between Each Evaluator during the Cognitive Walkthrough
The total number of identified issues in the Cognitive Walkthrough method from all evaluators is 34 problem items.Due to the presence of several identical issues found by each evaluator, the mapping of the problems was conducted.From the problem mapping, a total of 23 problem items were successfully obtained.

Frequency of Problem Found in Cognitive Walkthrough
The number of problems that were successfully identified by the evaluators using the Cognitive Walkthrough method before mapping is 34 items.Here is the frequency of occurrence of problems for each thought-provoking question:

Problem Mapping between Each Evaluator in Heuristic Evaluation
The total number of identified issues in the Heuristic Evaluation method from all evaluators is 63 items.Due to the presence of several identical issues found by each evaluator, the mapping of the problems was conducted.From the problem mapping, a total of 51 problem items were successfully obtained.

Frequency of Problem Found in Heuristic Evaluation
Based on the mode value of the heuristic number and the average severity rating determined in the previous analysis of Heuristic Evaluation Phase 2, the total problems and average severity rating for each heuristic number can be seen in Table 3.A comparison between the frequency of problems found and the average severity rating for each heuristic number can be seen in Figure 3.

Figure 3.
Comparison of average severity rating to total problems for each heuristic number   Caroline, Helen Sastypratiwi, at all | 1149

Similar Problem Found between Cognitive Walkthrough and Heuristic Evaluation
Based on the findings from the Cognitive Walkthrough and Heuristic Evaluation methods, several similar problems were found between the problem findings of Cognitive Walkthrough and Heuristic Evaluation.The similarities between these problems can be seen in Table 4.There is no option to view the password before logging in.
1. Users are not provided with the option to view the entered password.
1. Users are not given the option to view the entered password. 6 There is no help menu available if users forget their password.
1.There is no "Forgot Password" option available for students who forget their SPOTA account password.

There is no "Forgot
Password" option available for students who forget their SPOTA account password. 7 There is no highlight on the searched word in the "Penawaran Judul" tab.
1.In the search results table, there is no highlight on the inputted word/phrase in the search column.

In the search results table
for the title offers, there is no highlight on the word/phrase entered in the search field.

8
There is no information provided regarding the selection of supervisor 1-4 1.There is no information provided for supervisor 1, 2, 3, or 4 in the supervisor dropdown menu.
1.In the Pre-Outline upload section, there is no textview that provides information on which dropdown is intended for supervisor 1-4.

9
The page appears blank when clicking on Logout.
1.After logging out, it leads to an empty page.
1.A blank yellow-colored page appears after logging out.

10
The search button cannot be found.
1.The search button is not found when searching for offered titles. 1.In the Profile section, the floating helper text for the password is positioned above, which is unusual.
14 The use of the word "Upload" is not suitable.

The phrase "Upload Pra
Outline" doesn't feel suitable.
1.In the "Upload Pra Outline" section, the use of the word "Upload" should be replaced with "Ajukan" or "Ajukan Pra Outline".

15
The ambiguity of buttons that can be hovered over or not.
1.It is difficult to identify the button for viewing announcement details.
1.There are multiple instances where informational elements appear as buttons and vice versa.

Improvement Recommendation
Recommendations for improvement will be developed for each previously found issue.The types of improvement recommendations will be divided into two categories: that is system improvements and interface improvements.Interface recommendations will be provided in relation to the SPOTA UNTAN interface, while system recommendations will be given in relation to the functionality of SPOTA UNTAN.The improvement recommendations will be based on the 10 principles of heuristic evaluation, as well as suggestions provided by the evaluators during the problem identification process.

Discussion
User experience refers to the overall process of a user interaction with a product or service and their response to that experience [5].It is also defined as an individual perception and response resulting from the use and/or anticipation of using a product, system, or service [6].Several factors contribute to achieving a good user experience, that is having alignment to the product features with user needs, ensuring ease to use, especially during the first-time usage, to leave a positive impression, as well as the capability of the product or service to assist users in completing their tasks [7].Positive user experience helps people work faster and reduces errors when using a product.Improving the efficiency of the product enhances productivity [8].The improvement of user experience in SPOTA UNTAN also means increasing the capabilities, comfort, and efficiency of students in completing their final tasks or assignments.
The objective of the Heuristic Walkthrough method is to identify the existing issues in SPOTA UNTAN.Therefore, this study can serve as a guideline for designing the interface of SPOTA UNTAN.The research findings and analysis will offer valuable insights into addressing interface problems, enhancing user experience, and optimizing the usability of SPOTA UNTAN.By following the guidelines derived from this research, designers and developers can make informed decisions to create a more user-friendly and efficient interface, ultimately benefiting the users of SPOTA UNTAN.Based on the sources of problem findings, there are a of 15 problem findings from cognitive walkthrough and heuristic evaluation combined.Specifically, there are 2 problem findings from cognitive walkthrough and 22 problem findings from heuristic evaluation.Upon completing the Heuristic Walkthrough evaluation, the evaluators will provide recommendations for each identified problem in SPOTA UNTAN.These recommendations will offer practical solutions and improvements to address the interface issues found during the evaluation.The goal is to suggest design changes or enhancements that align with usability principles and best practices, thereby improving the overall user experience of SPOTA UNTAN.

CONCLUSION
The research has resulted in 39 recommendations for improvement.There are 34 recommendations focus on interface enhancements, while 5 recommendations relate to functional feature improvements.From all of those 34 interface-related recommendations, the suggested improvements include: adjusting the size of buttons and textfields, selecting appropriate colors that reflect the urgency of buttons or text, rearranging certain buttons and menus, improving helper text and placeholders that will assist users in filling out fields, rectifying information design that should not be hoverable, define hierarchy for text or buttons, adding tooltips to icons for better understanding of their purpose, implementing pagination.These recommendations aim to address specific issues identified during the evaluation process and provide actionable steps for enhancing the user interface of SPOTA.By implementing these recommendations, it is expected that the overall user experience of SPOTA will be improved, resulting in a more userfriendly and efficient system.

Figure 1 .
Figure 1.Flowchart of research process

Figure 1 .
Figure 1.Most frequently accessed page URL data of SPOTA UNTAN

Table 2 .
Frequency of problem found in cognitive walkthrough.

Table 3 .
Frequency of problem found in heuristic number

Table 4 .
Similar problems were found between cognitive walkthrough and heuristic evaluation.