Evaluation of User Experience in Integrated Learning Information Systems Using User Experience Questionnaire (UEQ)

Information and Communication Technology (ICT) has a role in assisting in monitoring and evaluating the learning process in schools. Integrated Learning Information System is the application of an integrated information system to improve the quality of academic and learning services accessed by all school members (teachers, homeroom teachers, foundations, and students) and parents/guardians of students. The learning information system focuses on making it easier to monitor and evaluate student academic and learning outcomes. User experience measurement is carried out to determine the extent to which the success of user understanding using the system so that the purpose of using the system can be achieved. This study aims to analyze the user experience of the Integrated Learning Information System using the User Experience Questionnaire (UEQ) method. This study uses quantitative data by distributing online questionnaires to 50 students. The results of the evaluation of user experience on learning information systems using the UEQ method state that attractiveness is 2.121, perspicuity is 2.152, efficiency is 2.319, dependability is 1.505, stimulation is 1.716, and novelty is 1.020. This proves that all aspects have positive values. The benchmark results showed that the attractiveness, perspicuity, efficiency, and stimulation aspects were included in the excellent criteria, while the dependability aspect was included in the good criteria and novelty included in the above average criteria.


INTRODUCTION
Information and Communication Technology (ICT) is any form of technology that can help generate and manipulate, communicate, store, and convey information. ICT is a significant need for school management to improve efficiency and productivity in the learning process. ICT plays a role in monitoring and evaluating the learning process in schools. The school management uses the implementation of monitoring and evaluation of the learning process to support decision-making in improving the quality of learning [1].
Integrated Learning Information System is the application of an integrated information system to improve the quality of academic and learning services accessed by all school members (teachers, homeroom teachers, foundations, and students) and parents/guardians of students. The Integrated Learning Information System focuses on making it easier to monitor and evaluate student academic and learning outcomes. User experience measurement is needed to determine the extent to which the success of user understanding in using information systems so that the purpose of using the system can be achieved. One of the success factors of an information system is that it can provide an experience of convenience and satisfaction for users when using software applications [2] [3]. User experience is a statement of feelings or experiences that users get when using software applications [4][5] [6]. A user interface that is too complex makes it difficult for users to use it, so it poses a risk of errors in use [7] [8].
This study aims to analyze the user experience of the Integrated Learning Information System using the User Experience Questionnaire (UEQ) method. UEQ is one of the most widely used usability testing measurement methods to evaluate user experience [9]. UEQ aims to obtain a comprehensive and pragmatic impression of usability and experience through a subjective quality assessment survey [7] [10]. The measurement of the UEQ method involves attractiveness, perspicuity, efficiency, dependability, stimulation, and novelty. UEQ measures technical and non-technical aspects of the user's emotions or perceptions of pleasure. UEQ can be used to perform measurements directly and quickly related to UX [12][13] [14].
UEQ can provide comprehensive user experience measurement results related to usability and experience aspects and is equipped with analytical tools in the form of Data Analysis Tools to explain the evaluation results [8] accurately. The results of the UEQ measurement can be used as a reference for improving the quality of the user interface. With the results of the UEQ method, it is possible to make predictions about areas where improvement will have a high impact [15].

METHODOLOGY
This research was conducted using the User Experience Questionnaire (UEQ) method through several stages, as shown in Figure 1.

Literature Study
This research went through the initial stage, namely conducting a literature study on the measurement of usability and experience. Literature study reviews previous research and reference books to obtain theories supporting user experience analysis.

Method Determination
The next stage in this research is determining the user experience method. Based on the results of the literature study that has been carried out, this study uses the User Experience Questionnaire (UEQ) method to analyze user experience in the Integrated Learning Information System. The study used the UEQ instrument consisting of 26 opposite items measured on a scale of one to seven on each item. The entire UEQ instrument in the study can be seen in Figure 2.

Data collection
UEQ evaluation is carried out for users of the Integrated Learning Information System. This study uses quantitative data by distributing online questionnaires to 50 students, making the UEQ questionnaire using Google Forms to facilitate the distribution of the questionnaire. The questionnaire contains 26 question points that refer to the UEQ questionnaire instrument.

Measurement of User Experience Questionnaire (UEQ)
The UEQ method is one of the usability testing measurement methods to obtain a comprehensive and pragmatic impression regarding usability and experience aspects through a subjective quality assessment survey. The evaluation results using the UEQ make it possible to predict areas where improvement will have a high impact. There are six scales with a total of 26 elements that are categorized based on the measurement scales contained in the UEQ: a. Attractiveness: How much the system attracts users. For example, likes or dislikes. b. Perspicuity: How much the system provides clarity to users. For example, easy-to-understand or difficult to understand c. Efficiency: How much the system provides efficiency to the user to complete the task. For example: sooner or later, practical or impractical. d. Dependability: How much the system provides accuracy to the user in controlling interactions with the system. Examples: predictable or unpredictable, favor or hinder. e. Stimulation: How much the system motivates users to use the product.
For example: useful or less useful, interesting or unattractive. f. Novelty: How much the system provides users with novelty value. For example: creative or not creative, conservative or innovative. Arista Pratama, Asif Faroqi, at all | 1023

RESULTS AND DISCUSSION
The results and discussion stages explain the data processing from the UEQ questionnaire obtained from all respondents.

Data Analytics
The UEQ questionnaire data analysis was done using the UEQ Data Analysis Tools. Data analysis produces an average value representing the user's system perception. Respondent data obtained from the questionnaire results were entered in the UEQ Data Analysis Tools. The UEQ Data Analysis Tools has several stages:

Data Transformed
Respondent data entered into excel will undergo a data conversion process; namely, the order of values (positive right and negative left) is randomized in the questionnaire to minimize the tendency of answers, as shown in Figure 3. Figure 3 shows the value order process used to minimize the tendency of the answers for each item. The converted data will produce an average value per person with each grouping based on aspects. Here's the formula for the average data conversion:

Results UEQ
The result of the average data conversion will be recalculated to get the main result (result). The result is the main result of the UEQ, which is the benchmark for the following calculation, namely the benchmark. The overall scale and assumption Values for each category can be seen in Table 1. The interpretation of each category is: 1) Excellent: the highest score of the product range is 10%, 2) Good: 10% higher score products, while 50% lower, 3) Above Average: 25% higher product dataset, 50% lower, 4) Below Average: 50% higher product dataset, 25% lower, 5) Bad: 25% have a low score

Results Analysis
The analysis of the results discusses the overall UEQ calculation, which results in the mean, variance, and standard deviation for 26 items. The value of each item Arista Pratama, Asif Faroqi, at all | 1025 has a range above 1, while the novelty item has a value below 1, which is 0.9. The assessment shows that each question item has a good score.
Based on Figure 3  The full scale will be reprocessed to determine the pragmatic and hedonic qualities. Pragmatic quality consists of perspicuity, efficiency, and dependability. Hedonic consists of stimulation and novelty. Assessment for pragmatic and hedonic can be seen in Table 2.  Based on the processing results from Table 2, the highest value is Attractiveness, which is 2.121. In the pragmatic quality aspect, the highest value is efficiency, which is 2.319. This shows that the Integrated Learning Information System is an attractive and convenient system for users, and users can complete tasks quickly due to attractiveness and efficiency factors, getting positive scores. The user can control the interaction with the system, which is shown in the aspect of dependability which has a value of 1.505. The perspicuity aspect gets a positive score of 2.152. This shows that the system is easy for users to understand. The test results on the hedonic quality aspect have a value of 1.37. Stimulation and novelty are aspects that enter into hedonic quality. The result of calculating the UEQ value for the stimulation aspect is 1.716. This shows that the Integrated Learning Information System application is useful for users and motivates users to use the system. The UEQ value for the novelty aspect is 1.020, indicating the system design has novelty.
Based on the results of this study, it was found that the evaluation results for all aspects had a positive value, where all UX aspects scored above 0.8. This follows that the impression value between -0.8 and 0.8 is a normal evaluation value, a value > 0.8 is a positive value, and a value < -0.8 is a negative value [6]. Another analysis that will be carried out is a benchmark analysis. The analysis compares this study with similar studies [7].  Table 3, it can be seen that the highest rating is on the efficiency scale, and the lowest value is on the novelty scale. Therefore the novelty scale needs to be improved in terms of user experience. The form of the benchmark data set results is then processed into a graphic diagram to make it easier to observe the assessment of each scale. The chart of the results of the benchmark data set can be seen in Figure 4.

CONCLUSION
This study shows the results of the Integrated Learning Information System evaluation, which was carried out using the User Experience Questionnaire (UEQ) showed that the attractiveness, perspicuity, efficiency, and stimulation aspects were included in the excellent criteria, while the dependability aspect was included in the good criteria and novelty included in the above average criteria. In the novelty aspect with a normal impression, attention must be paid to improve it so that the Integrated Learning Information System application in the future becomes better.