ISSN: 2226-6348
Open access
While Virtual and Augmented Reality (VR/AR) technologies have advanced English language learning, existing adaptive systems rely on fragmented analyses of learner interactions. Current environments process speech, gaze, or gesture in isolation, leading to static instructional designs, limited personalization, and reduced pedagogical responsiveness. To address this gap, this paper proposes an AI-driven multi-modal adaptive framework that integrates Natural Language Processing (NLP), Reinforcement Learning (RL), and multi-modal behavioural fusion for personalized English language acquisition. Grounded in Self-Determination Theory, Embodied Cognition Theory, Scaffolding Theory, and Cognitive Load Theory, the conceptual architecture dynamically synthesizes concurrent behavioural streams to drive real-time, responsive pedagogical interventions. The proposed model overcomes single-channel limitations by translating multi-modal learner data into adaptive instructional decisions. Theoretically, this framework delivers superior intelligent scaffolding, fosters learner autonomy by adapting to user agency, and manages cognitive load through optimized content delivery within immersive environments. Ultimately, this framework shifts VR/AR language learning from static interfaces to highly responsive, personalized ecosystems, offering a robust blueprint for next-generation educational technologies, future empirical validation, and higher education implementation.
Aleven, V., McLaren, B. M., Sewall, J., Velsen, M. van, Popescu, O., Demi, S., Ringenberg, M., & Koedinger, K. R. (2016). Example-Tracing Tutors: Intelligent tutor development for non-programmers. International Journal of Artificial Intelligence in Education, 26(1), 224–269. https://doi.org/10.1007/s40593-015-0088-2
Al-Jarf, R. (2021). Feasibility of digital multimedia language labs for interpreting instruction as perceived by interpreting instructors in Saudi Arabia. International Journal of Linguistics Literature & Translation, 4(4), 70–79. https://doi.org/10.32996/ijllt.2021.4.4.8
Aziz, N. A. N. B., Hanif, Z. S. A. B. M., Kasim, V. N. B. M., Baharin, N. N. B., & Manap, M. R. (2025). Tackling presentation anxiety among ESL university students. International Journal of Research and Innovation in Social Science, 9(22), 224–229. https://doi.org/10.47772/ijriss.2025.922ileiid0023
Brenes, C. A. N. (2011). The language laboratory and the EFL course. Actualidades Investigativas en Educación, 6(2). https://doi.org/10.15517/aie.v6i2.9209
Bulathwela, S., Niekerk, D. van, Shipton, J., Pérez?Ortiz, M., Rosman, B., & Shawe?Taylor, J. (2025). TrueReason: An exemplar personalised learning system integrating reasoning with foundational models. In ArXiv.org. https://doi.org/10.48550/arxiv.2502.10411
Chen, Y., Kao, T. C., & Sheu, J. (2003). A mobile learning system for scaffolding bird watching learning. Journal of Computer Assisted Learning, 19(3), 347–359. https://doi.org/10.1046/j.0266-4909.2003.00036.x
Chen, Y. T., & Dang, J. (2025). Developing a virtual reality system integrated with large language models for real-time evaluation and feedback to improve public speaking skills. International Journal of Research and Innovation in Social Science, 165–176. https://doi.org/10.5121/csit.2025.151713
Chowdhury, S. P., Zouhar, V., & Sachan, M. (2024). AutoTutor meets large language models: A language model tutor with rich pedagogy and guardrails. 5–15. https://doi.org/10.1145/3657604.3662041
Chun, D. M., Karimi, H., & Sañosa, D. J. (2022). Traveling by headset. CALICO Journal, 39(2), 129–149. https://doi.org/10.1558/cj.21306
Deci, E. L., Vallerand, R. J., Pelletier, L. G., & Ryan, R. M. (1991). Motivation and Education: The Self-Determination Perspective. Educational Psychologist, 26(3), 325–346. https://doi.org/10.1207/s15326985ep2603&46
Endra, B., & André, E. (2014). Integration of cultural factors into the behavioral models of virtual characters. In OPUS (Augsburg University) (pp. 227–251). Augsburg University. https://doi.org/10.1017/cbo9780511844492.010
Fernandino, L., Binder, J. R., Desai, R. H., Pendl, S. L., Humphries, C., Gross, W. L., Conant, L. L., & Seidenberg, M. S. (2015). Concept representation reflects multimodal abstraction: A framework for embodied semantics. Cerebral Cortex, 26(5), 2018–2034. https://doi.org/10.1093/cercor/bhv020
Galvis, Á. (2018). Supporting decision-making processes on blended learning in higher education: literature and good practices review. International Journal of Educational Technology in Higher Education, 15(1). https://doi.org/10.1186/s41239-018-0106-1
Hajji, M. E., Baha, T. A., Berka, A., Nacer, H. A., Aouifi, H. E., & Es-Saady, Y. (2025). An architecture for intelligent tutoring in virtual reality: Integrating LLMs and multimodal interaction for immersive learning. Information, 16(7), 556–556. https://doi.org/10.3390/info16070556
Herawati, A. (2023). Implementing personalized learning in universities classrooms: Lecturers’ challenges and perceptions. Humaniora, 14(2), 167–172. https://doi.org/10.21512/humaniora.v14i2.8890
Huang, X., Zou, D., Cheng, G., & Xie, H. (2021). A systematic review of AR and VR enhanced language learning. Sustainability, 13(9), 4639–4639. https://doi.org/10.3390/su13094639
Ipsita, A., Patel, M., Unmesh, A., & Ramani, K. (2024). Authoring instructional flow in VR learning units to promote outcome-oriented learning. Computers & Education X Reality, 5, 100074–100074. https://doi.org/10.1016/j.cexr.2024.100074
Ironsi, C. S. (2023). Investigating the use of virtual reality to improve speaking skills: insights from students and teachers. Smart Learning Environments, 10(1). https://doi.org/10.1186/s40561-023-00272-8
Kaplan?Rakowski, R., & Gruber, A. (2023). The impact of high-immersion virtual reality on foreign language anxiety. Smart Learning Environments, 10(1). https://doi.org/10.1186/s40561-023-00263-9
Kishor, I., Mamodiya, U., Almaayah, M., Alqutaish, A., Shehab, R., & Aldhyani, T. H. H. (2025). Agentic AI-Enhanced Virtual Reality for adaptive immersive learning environments. Mesopotamian Journal of Computer Science, 2025, 398–416. https://doi.org/10.58496/mjcsc/2025/026
Koning, B. B. de, & Tabbers, H. K. (2011). Facilitating understanding of movements in dynamic visualizations: An embodied perspective. Educational Psychology Review, 23(4), 501–521. https://doi.org/10.1007/s10648-011-9173-8
Lampropoulos, G. (2025). Augmented Reality, Virtual Reality, and Intelligent Tutoring Systems in Education and Training: A Systematic Literature review. Applied Sciences, 15(6), 3223. https://doi.org/10.3390/app15063223
Ludwig-Hardman, S., & Dunlap, J. C. (2003). Learner support services for online students: Scaffolding for success. The International Review of Research in Open and Distributed Learning, 4(1). https://doi.org/10.19173/irrodl.v4i1.131
Mukherjee, P. K. (2026). Re-examining technology integration in education: Cognitive foundations, pedagogical implications, and contextual constraints. International Journal of Novel Trends and Innovation, 4(3). https://doi.org/10.56975/ijnti.v4i3.231769
Ortiz, L. G., Clougher, D., Anderson, T., & Maina, M. F. (2021). IDEAS for transforming higher education: An overview of ongoing trends and challenges. The International Review of Research in Open and Distributed Learning, 22(2), 166–184. https://doi.org/10.19173/irrodl.v22i2.5206
Pappas, M. A., & Drigas, A. (2016). Incorporation of Artificial Intelligence tutoring techniques in Mathematics. International Journal of Engineering Pedagogy (iJEP), 6(4), 12–12. https://doi.org/10.3991/ijep.v6i4.6063
Ruan, S., & Lu, K. (2025). Adaptive deep reinforcement learning for personalized learning pathways: A multimodal data-driven approach with real-time feedback optimization. Computers and Education Artificial Intelligence, 9, 100463–100463. https://doi.org/10.1016/j.caeai.2025.100463
Sajja, R., Sermet, Y., Cwiertny, D. M., & Demir, ?. (2025). Integrating AI and learning analytics for data-driven pedagogical decisions and personalized interventions in education. Technology Knowledge and Learning. https://doi.org/10.1007/s10758-025-09897-9
Skaramagkas, V., Giannakakis, G., Ktistakis, E., Manousos, D., Karatzanis, I., Tachos, N. S., Tripoliti, E. E., Marias, K., Fotiadis, D. I., & Tsiknakis, M. (2021). Review of eye tracking metrics involved in emotional and cognitive processes. IEEE Reviews in Biomedical Engineering, 16, 260–277. https://doi.org/10.1109/rbme.2021.3066072
Skulmowski, A., & Xu, K. M. (2021). Understanding cognitive load in digital and online learning: A new perspective on extraneous cognitive load. Educational Psychology Review, 34(1), 171–196. https://doi.org/10.1007/s10648-021-09624-7
Son, J., Ruži?, N. K., & Philpott, A. (2023). Artificial intelligence technologies and applications for language learning and teaching. Journal of China Computer-Assisted Language Learning, 5(1), 94–112. https://doi.org/10.1515/jccall-2023-0015
Sukirman, S. (2023). Exploring the impacts of the covid-19 pandemic on English language teachers. Vision Journal for Language and Foreign Language Learning, 12(1), 1–16. https://doi.org/10.21580/vjv12i216998
Sun, L., Dong, H., & Zhang, X. (2023). Innovative solutions for language growth: the impact of problem-based learning via DingTalk on Chinese undergraduates’ business vocabulary amid COVID-19. Frontiers in Psychology, 14, 1289575–1289575. https://doi.org/10.3389/fpsyg.2023.1289575
Sweller, J., Merriënboer, J. J. G. van, & Paas, F. (2019). Cognitive architecture and instructional design: 20 Years Later. Educational Psychology Review, 31(2), 261–292. https://doi.org/10.1007/s10648-019-09465-5
Syed, T. A., Khan, S., Jan, S., & Nauman, M. (2025). Revolutionizing arabic learning: GNT and AI-enhanced Metaverse environments. Frontiers in Computer Science, 7. https://doi.org/10.3389/fcomp.2025.1626092
Yan, L., Echeverría, V., Jin, Y., Fernandez-Nieto, G. M., Zhao, L., Li, X., Alfredo, R., Swiecki, Z., Gaševi?, D., & Martínez?Maldonado, R. (2024). Evidence?based multimodal learning analytics for feedback and reflection in collaborative learning. British Journal of Educational Technology, 55(5), 1900–1925. https://doi.org/10.1111/bjet.13498
Yan, W., & Lowell, V. L. (2025). The Evolution of Virtual Reality in foreign language education: From Text-Based MUDs to AI-Enhanced Immersive Environments. TechTrends, 69(4), 853–858. https://doi.org/10.1007/s11528-025-01086-w
Yusuf, H., Money, A., & Daylamani-Zad, D. (2025). Towards reducing teacher burden in Performance-Based assessments using aivaluate: an emotionally intelligent LLM-Augmented pedagogical AI conversational agent. Education and Information Technologies, 30(17), 24649–24693. https://doi.org/10.1007/s10639-025-13755-7
Zawacki?Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – where are the educators? International Journal of Educational Technology in Higher Education, 16(1). https://doi.org/10.1186/s41239-019-0171-0
Hon, K. K. L. A., & Valentine, A. B. (2026). Integrating Multi-Modal Machine Learning into Immersive Pedagogy: A Conceptual Framework for AI-Driven Adaptive Scaffolding in VR/AR English Language Learning. International Journal of Academic Research in Progressive Education and Development, 15(4), 820–845.
Copyright: © 2026 The Author(s)
Published by HRMARS (www.hrmars.com)
This article is published under the Creative Commons Attribution (CC BY 4.0) license. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this license may be seen at: http://creativecommons.org/licences/by/4.0/legalcode