ACTE: A Pilot Feasibility Evaluation of a Mastery-Aware Task Recommender for Mobile Language Learning in Real-World Contexts

Authors

  • Yudhy Setyo Purwanto Diponegoro University; Institut Teknologi PLN, Indonesia
  • Rahmat Gernowo Diponegoro University, Indonesia
  • Dinar Mutiara Kusumo Nugraheni Diponegoro University, Indonesia
Pages Icon

DOI:

https://doi.org/10.63158/journalisi.v8i2.1554

Keywords:

mobile-assisted language learning (MALL), context-aware task recommendation, CEFR-based task design, mastery learning, situated learning

Abstract

Mobile-assisted language learning (MALL) apps often present generic activities that ignore the semantic meaning of real-world places and provide limited skill-specific, mastery-based progression. This pilot feasibility study introduces the Adaptive Contextual Task Engine (ACTE), a lightweight on-device recommender that personalizes tasks using location semantics, CEFR-aligned modules, mastery status, and performance timing. ACTE was evaluated with 10 university students aged 18–23 in a simulated café environment to balance ecological validity and experimental control. Participants completed three A2 speaking tasks, the System Usability Scale (SUS), and a five-item relevance questionnaire. Results showed a mean SUS score of 72.0, exceeding the benchmark of 68. Participants rated task appropriateness for the café at 4.3/5 and real-life usability at 4.7/5, while 90% agreed that the tasks reflected authentic language use. Qualitative feedback confirmed contextual authenticity but indicated the need for clearer scoring explanations. These findings suggest that ACTE offers a practical, privacy-conscious, and replicable framework for situated MALL by linking semantic place affordances with mastery-based progression in controlled real-world simulations.

Downloads

Download data is not yet available.

References

[1] B. Choi, “Playful Assessment: A Game-Based Approach to Assessing Teachers’ Competency for the Wise Integration of Technology in the Classroom,” University of Connecticut, Storrs, CT, USA, 2017.

[2] J. S. Brown, A. Collins, and P. Duguid, “Situated cognition and the culture of learning,” Educ. Res., vol. 18, no. 1, pp. 32–42, 1989, doi: 10.3102/0013189X018001032.

[3] B. Morgan, “Situated cognition and the study of culture: An introduction,” Poet. Today, vol. 38, no. 2, pp. 213–233, 2017, doi: 10.1215/03335372-3868421.

[4] R. Pederson, “Situated learning: Rethinking a ubiquitous theory,” J. Asia TEFL, vol. 9, no. 2, pp. 123–148, 2012.

[5] R. Godwin-Jones, “Emerging technologies: Mobile apps for language learning,” Lang. Learn. Technol., vol. 15, no. 2, pp. 2–11, 2011.

[6] R. Godwin-Jones, “Emerging spaces for language learning: AI bots, ambient intelligence, and the metaverse,” Lang. Learn. Technol., vol. 27, no. 2, pp. 6–27, 2023, doi: 10.64152/10125/73501.

[7] R. Godwin-Jones, “Distributed agency in second language learning and teaching through generative AI,” Lang. Learn. Technol., vol. 28, no. 2, pp. 5–31, 2024, doi: 10.64152/10125/73570.

[8] A. Bahari, X. Zhang, and Y. Ardasheva, “Establishing a computer-assisted interactive reading model,” Comput. Educ., vol. 172, no. September 2020, p. 104261, 2021, doi: 10.1016/j.compedu.2021.104261.

[9] H. Robles, K. Burden, and K. Villalba, “A socio-cultural approach to evaluating and designing reading comprehension apps for language learning,” Int. J. Mob. Blended Learn., vol. 13, no. 1, pp. 18–37, 2021, doi: 10.4018/IJMBL.2021010102.

[10] Y. J. Lee and P. Roger, “Cross-platform language learning: A spatial perspective on narratives of language learning across digital platforms,” System, vol. 118, no. October, p. 103145, 2023, doi: 10.1016/j.system.2023.103145.

[11] A. Kukulska-Hulme and O. Viberg, “Mobile collaborative language learning: State of the art,” Br. J. Educ. Technol., vol. 49, no. 2, pp. 207–218, 2018, doi: 10.1111/bjet.12580.

[12] O. A. Al-Smadi, R. A. Rashid, H. Saad, Y. H. Zrekat, S. S. L. A. Kamal, and G. I. Uktamovich, “Artificial Intelligence for English Language Learning and Teaching: Advancing Sustainable Development Goals,” J. Lang. Teach. Res., vol. 15, no. 6, pp. 1835–1844, 2024, doi: 10.17507/jltr.1506.09.

[13] M. Mihaylova, S. Gorin, T. Reber, and N. Rothen, “A meta-analysis on mobile assisted language learning applications reveals moderate learning benefit and significant publication bias,” pp. 1–52, 2020, [Online]. Available: https://osf.io/preprints/ux93y/

[14] Y. Zhou and M. Zhou, “A meta-analysis on mobile-assisted vocabulary learning: Do mobile applications help?,” ReCALL, vol. 38, pp. 75–93, 2025, doi: 10.1017/S0958344025100335.

[15] Y. Bai, “A Mixed Methods Investigation of Mobile-Based Language Learning on EFL Students’ Listening, Speaking, Foreign Language Enjoyment, and Anxiety,” SAGE Open, vol. 14, no. 2, pp. 1–19, 2024, doi: 10.1177/21582440241255554.

[16] K. Tabuchi, S. Kobayashi, S. T. Fukuda, and Y. Nakagawa, “A Case Study on Reducing Language Anxiety and Enhancing Speaking Skills Through Online Conversation Lessons,” Technol. Lang. Teach. Learn., vol. 6, no. 3, pp. 1–21, 2024, doi: 10.29140/tltl.v6n3.1497.

[17] B. Olovson and M. Seigler, “Overcoming Skill-Specific Language Learning Anxiety: Research-Based Tools,” in Room for All at the Table, 1st ed., B. M. Burke, Ed., Ashland, VA 23005: CSCTFL, 2020, pp. 47–74.

[18] Z. Zhou, “A Systematic Literature Review on the use of Mobile-assisted Language Learning (MALL) for Enhancing Speaking Skills in Chinese EFL context,” Int. J. Front. Sociol., vol. 3, no. 15, pp. 12–24, 2021, doi: 10.25236/ijfs.2021.031502.

[19] R. Li, “Effects of mobile-assisted language learning on foreign language learners’ speaking skill development,” Lang. Learn. Technol., vol. 28, no. 1, pp. 1–26, 2024, doi: 10.64152/10125/73553.

[20] T. Jantakoon, T. Jantakun, K. Jantakun, W. Pongpanich, R. Pasmala, P. Wannapiroon, and P. Nilsook, "The effectiveness of artificial intelligence in English instruction for speaking and listening skills: A meta-analysis," Contemp. Educ. Technol., vol. 17, no. 4, p. ep596, 2025, doi: 10.30935/cedtech/17310.

[21] S. Bourekkache and O. Kazar, “Mobile and adaptive learning application for english language learning,” Int. J. Inf. Commun. Technol. Educ., vol. 16, no. 2, pp. 36–46, 2020, doi: 10.4018/IJICTE.2020040103.

[22] B. Garg and D. S. Pant, “Effectiveness Of Adaptive Mobile-Assisted Language Learning (Mall) For Efl Vocabulary Acquisition Among Undergraduate Students,” Int. J. Environ. Sci., vol. 11, no. 18, pp. 2223–2231, 2025, doi: 10.64252/pqpf2j74.

[23] A. Bahari, “Affordances and challenges of technology-assisted language learning for motivation: A systematic review,” Interact. Learn. Environ., vol. 31, no. 9, pp. 5853–5873, 2023, doi: 10.1080/10494820.2021.2021246.

[24] A. Bobunova, M. Sergeeva, and E. Notina, Integrating Computer-Assisted Language Learning into ESL Classroom: Formation of Moral and Aesthetic Values. 2021. doi: 10.18178/ijiet.2021.11.1.1484.

[25] E. S. H. Haataja, A. Tolvanen, H. Vilppu, M. Kallio, J. Peltonen, and R. L. Metsäpelto, “Measuring higher-order cognitive skills with multiple choice questions –potentials and pitfalls of Finnish teacher education entrance,” Teach. Teach. Educ., vol. 122, 2023, doi: 10.1016/j.tate.2022.103943.

[26] A. T. Lidyasari, I. Rachmawati, A. Da Costa, and P. Wanyi, “How are the Cognitive, Affective, and Psychomotor Levels of Primary School Learners Living in Suburban Area of Yogyakarta based on Career Development?,” J. Prima Edukasia, vol. 10, no. 2, pp. 130–137, 2022, doi: 10.21831/jpe.v10i2.48061.

[27] T. H. Phuc and T. T. Nghi, “Examining the Impact of Mobile Apps on Language Teaching and Learning in a Public University: An Experimental Study,” Int. J. Linguist. Lit. Transl., vol. 3, no. 11, pp. 55–67, 2023, doi: 10.32996/ijllt.2023.6.6.12.

[28] N. Figueras, D. Little, and B. O’Sullivan, Aligning Language Education with the CEFR: A Handbook, vol. 5, no. April. 2022. doi: 10.37546/jaltsig.cefr5-1.

[29] A. Z. Amiruddin, M. T. A. Ghani, W. A. A. W. Daud, N. H. Hussein, N. H. M. Yamin, and Y. S. Abdullah, “Empowering language learners: Innovations in CEFR-based language learning applications,” Edelweiss Appl. Sci. Technol., vol. 9, no. 2, pp. 352–359, 2025, doi: 10.55214/25768484.v9i2.4486.

[30] W. C. Fang, H. C. Yeh, B. R. Luo, and N. S. Chen, “Effects of mobile-supported task-based language teaching on EFL students’ linguistic achievement and conversational interaction,” ReCALL, vol. 33, no. 1, pp. 71–87, 2021, doi: 10.1017/S0958344020000208.

[31] J. Wu, H. Jiang, and S. Chen, “Augmented reality technology in language learning: A meta-analysis,” Lang. Learn. Technol., vol. 28, no. 1, pp. 1–23, 2024, doi: 10.64152/10125/73596.

[32] B. Chen, Y. Wang, and L. Wang, “The Effects of Virtual Reality-Assisted Language Learning: A Meta-Analysis,” Sustain., vol. 14, no. 6, 2022, doi: 10.3390/su14063147.

[33] E. Ericsson and S. Johansson, “English speaking practice with conversational AI: Lower secondary students’ educational experiences over time,” Comput. Educ. Artif. Intell., vol. 5, no. April, p. 100164, 2023, doi: 10.1016/j.caeai.2023.100164.

[34] B. Khosrawi-Rad, H. Rinn, D. Augenstein, D. Markgraf, and S. Robra-Bissantz, “Designing Pedagogical Conversational Agents in Virtual Worlds,” Lect. Notes Informatics (LNI), Proc. - Ser. Gesellschaft fur Inform., vol. P-338, pp. 181–186, 2023, doi: 10.18420/delfi2023-29.

[35] R. R. Divekar, H. Lepp, P. Chopade, A. Albin, D. Brenner, and V. Ramanarayanan, “Conversational Agents in Language Education: Where They Fit and Their Research Challenges,” Commun. Comput. Inf. Sci., vol. 1499 CCIS, pp. 272–279, 2021, doi: 10.1007/978-3-030-90179-0_35.

[36] M. Pegrum and A. Palalas, "Attentional literacy as a new literacy: Helping students deal with digital disarray," Can. J. Learn. Technol., vol. 47, no. 2, pp. 1–18, 2021, doi: 10.21432/CJLT28037.

[37] M. Pegrum and Y. J. Lan, “Extended reality (XR) in language learning: Developments and directions,” Lang. Learn. Technol., vol. 27, no. 3, pp. 1–5, 2023, doi: 10.64152/10125/73528.

[38] Y. M. Fromm and D. Ifenthaler, “Designing adaptive learning environments for continuing education: Stakeholders’ perspectives on indicators and interventions,” Comput. Hum. Behav. Reports, vol. 16, no. August, 2024, doi: 10.1016/j.chbr.2024.100525.

[39] E. A. Oliveira, P. de Barba, and L. Corrin, “Enabling Adaptive, Personalised and Context-aware Interaction in a Smart Learning Environment: Piloting the Icollab System,” Australas. J. Educ. Technol., vol. 37, no. 2, pp. 1–23, 2021, doi: 10.14742/AJET.6792.

[40] J. P. Lantolf, M. E. Poehner, and S. L. Thorne, “Sociocultural Theory and L2 Development,” Theor. Second Lang. Acquis., pp. 223–247, 2020, doi: 10.4324/9780429503986-10.

[41] J. P. Lantolf, “Sociocultural Theory Learning Language Second Introduction to the Special Issue,” Natl. Fed. Mod. Lang. Teach., vol. 78, no. 4, pp. 418–420, 2008.

[42] J. P. Lantolf, “The sociocultural approach to second language acquisition,” 2nd ed. New York, NY, USA: Routledge, 2010, pp. 24–47.

[43] J. P. Lantolf and M. E. Poehner, “Sociocultural theory and classroom second language learning in the East Asian context: Introduction to the special issue,” Mod. Lang. J., vol. 107, pp. 3–23, 2023, doi: 10.1111/modl.12816.

[44] C. Kramsch and Contributors, Language Acquisition and Language Socialization: Ecological Perspectives. London, U.K.: Continuum, 2003.

[45] S. V. Steffensen and C. Kramsch, “The Ecology of Second Language Acquisition and Socialization,” in Language Socialization, Encyclopedia of Language and Education, 2017, pp. 1–16. doi: 10.1007/978-3-319-02327-4_2-1.

[46] H. T. Nguyen Tran, W. Jou She, S. Kajimura, and Y. Shibuya, “Assessing Notification Timing Strategies for Improved Micro-Learning Engagement,” IEEE Access, vol. 13, pp. 64534–64545, 2025, doi: 10.1109/ACCESS.2025.3559521.

[47] D. Rohrer, “Interleaving helps students distinguish among similar concepts . Educational Psychology Interleaving Helps Students Distinguish among Similar Concepts,” Educ. Psychol. Rev., vol. 24, pp. 355–367, 2012.

[48] A. R. Hevner, S. T. March, J. Park, and S. Ram, “Design science in information systems research,” MIS Q. Manag. Inf. Syst., vol. 28, no. 1, pp. 75–105, 2004, doi: 10.2307/25148625.

[49] A. K. Carstensen and J. Bernhard, “Design science research–a powerful tool for improving methods in engineering education research,” Eur. J. Eng. Educ., vol. 44, no. 1–2, pp. 85–102, 2019, doi: 10.1080/03043797.2018.1498459.

[50] K. Peffers, T. Tuunanen, M. A. Rothenberger, and S. Chatterjee, “A design science research methodology for information systems research,” J. Manag. Inf. Syst., vol. 24, no. 3, pp. 45–77, 2007, doi: 10.2753/MIS0742-1222240302.

[51] B. Kremmel, K. Eberharter, E. Konrad, and ..., “The CEFR Companion Volume: Opportunities and challenges for language assessment,” Didakt. slawischer …, no. 3, 2023, doi: 10.48789/2023.

[52] K. Leitner and B. Kremmel, “Avoiding Scoring Malpractice: Supporting Reliable Scoring of Constructed-Response Items in High-Stakes Exams,” Challenges in Language Testing Around the World. Springer Singapore, pp. 127–145, 2021. doi: 10.1007/978-981-33-4232-3_10.

[53] X. Chen et al., “Modeling question difficulty for unbiased cognitive diagnosis: A causal perspective,” Knowledge-Based Syst., vol. 294, no. March 2023, p. 111750, 2024, doi: 10.1016/j.knosys.2024.111750.

[54] N. U. C. Mustaffa and S. N. Sailin, “A Systematic Review of Mobile-Assisted Language Learning Research Trends and Practices in Malaysia,” Int. J. Interact. Mob. Technol., vol. 16, no. 5, pp. 169–198, 2022, doi: 10.3991/ijim.v16i05.28129.

[55] Q. Xu and J. C. Richardson, “The Impact of Social Media and Gamification of a Mobile Vocabulary Learning App: Self-Regulation and Learning Persistence,” Online Learn. J., vol. 28, no. 4, pp. 4–32, 2024, doi: 10.24059/olj.v28i4.4592.

[56] J. Brooke, “SUS: A ‘Quick and Dirty’ Usability Scale,” in Usability Evaluation In Industry, no. January 1996, 1996, pp. 189–194. doi: 10.1201/9781498710411-35.

[57] K. Ishaq, N. Azan, F. Rosdi, A. Abid, and Q. Ali, Usability of Mobile Assisted Language Learning App. 2020. doi: 10.14569/ijacsa.2020.0110145.

[58] V. Braun and V. Clarke, “Reflecting on reflexive thematic analysis,” Qual. Res. Sport. Exerc. Heal., vol. 11, no. 4, pp. 589–597, 2019, doi: 10.1080/2159676X.2019.1628806.

[59] G. Konstantinos, “Thematic analysis: A practical guide,” Eur. J. Psychother. Couns., vol. 26, no. 3–4, pp. 461–464, 2024, doi: 10.1080/13642537.2024.2391666.

[60] N. Tran, S. Kajimura, and Y. Shibuya, “Location- and Physical-Activity-Based Application for Japanese Vocabulary Acquisition for Non-Japanese Speakers,” Multimodal Technol. Interact., vol. 7, no. 3, 2023, doi: 10.3390/mti7030029.

[61] L. Hakimi, R. Eynon, and V. A. Murphy, The Ethics of Using Digital Trace Data in Education: A Thematic Review of the Research Landscape, vol. 91, no. 5. 2021. doi: 10.3102/00346543211020116.

[62] J. C. Dunn and C. Zimmer, “Self-determination theory,” Routledge Handb. Adapt. Phys. Educ., vol. 55, no. 1, pp. 296–312, 2020, doi: 10.4324/9780429052675-23.

Downloads

Published

2026-04-12

Issue

Vol. 8 No. 2 (2026): April

Section

Articles

License

Copyright (c) 2026 Journal of Information Systems and Informatics

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Authors Declaration

  1. The Authors certify that they have read, understood, and agreed to the Journal of Information Systems and Informatics (JournalISI) submission guidelines, policies, and submission declaration. The submission has been prepared using the provided template.
  2. The Authors certify that all authors have approved the publication of this manuscript and that there is no conflict of interest.
  3. The Authors confirm that the manuscript is their original work, has not received prior publication, is not under consideration for publication elsewhere, and has not been previously published.
  4. The Authors confirm that all authors listed on the title page have contributed significantly to the work, have read the manuscript, attest to the validity and legitimacy of the data and its interpretation, and agree to its submission.
  5. The Authors confirm that the manuscript is not copied from or plagiarized from any other published work.
  6. The Authors declare that the manuscript will not be submitted for publication in any other journal or magazine until a decision is made by the journal editors.
  7. If the manuscript is finally accepted for publication, the Authors confirm that they will either proceed with publication immediately or withdraw the manuscript in accordance with the journal’s withdrawal policies.
  8. The Authors agree that, upon publication of the manuscript in this journal, they transfer copyright or assign exclusive rights to the publisher, including commercial rights

How to Cite

[1]
Y. S. Purwanto, R. Gernowo, and D. M. K. Nugraheni, “ACTE: A Pilot Feasibility Evaluation of a Mastery-Aware Task Recommender for Mobile Language Learning in Real-World Contexts”, journalisi, vol. 8, no. 2, pp. 1640–1671, Apr. 2026, doi: 10.63158/journalisi.v8i2.1554.