Generative Artificial Intelligence Chatbot Integration for Virtual Reality Language Learning

Long Cheng^1,2, Joy Gisler², Kenli Lao², Andreas Kunz²

This paper describes the potential of integrating generative artifi-cial intelligence (GenAI) based chatbots into virtual reality (VR) for language learning. Existing VR and AI-based systems primarily focus on practicing vocabulary or written skills, often neglecting speaking practice. To address this gap, a VR-enabled conversation trainer, chatbot AI for VR (CHAI), is introduced. The system integrates speech-to-text, a large language model, text-to-speech, and an avatar to enable natural conversations with CHAI. It is evaluated for its language error detection capabilities in English and German. The results show high accuracy in detecting language errors and demonstrate substantial reproducibility in error detection.

¹RhySearch, Forschung und Innovationzentrum Rheintal, Werdenbergstrasse 4, 9471 Buchs Switzerland
²Institute for Machine Tools and Manufacturing, Swiss Federal Institute of Technology Zurich, Clausiusstrasse 33, Zurich Switzerland

Download Publication