Project 5 VR Agent

Project 5: Education

Together with partners such as SpaceBuzz and Zwijsen, we investigate how to effectively use virtual reality in education. Given the rapid changes in our society and the important role of technology in this process, the current education system is outdated and in dire need of change. The use of artificial intelligence in combination with virtual reality environments may offer a solution, especially in the embodiment of a virtual teacher.

Virtual reality in education

Partners

The project 

The behavior and even the appearance of the virtual teacher can be adapted to best suit the student's interests. In the project, we focus on improving the technology of avatars and to make them as natural as possible. This means that we are working on natural speech and behaviors, facial movements and appropriate expressions to go with a dialogue.

In this project, we investigate the use of a virtual teacher in primary education. We conduct studies in primary schools on the various aspects of the embodiment of a virtual teacher, such as speech, characteristics and contextual relevance. In a study on computer-generated speech in Dutch, we tested different Dutch voices made from varying amounts of training data. We evaluated the voice quality of the text-to-speech (TTS) models and also evaluated to what extent the quality of the voice affects how well students remember information. In another study, we explored the effects of combining TTS voices with VR agents, focusing on student perceptions and recall. Do students have a better perception of TTS voices when paired with a VR agent? 

In our most recent VR study, we delved deeper into the effects of content-matched VR environments and agent outfit on learning outcomes and experience. Do students show improved performance when the VR agent's outfit and the environment are tailored to match the learning content? We are currently expanding our research to investigate how agent interactivity influences students' learning outcomes and overall experience, aiming to optimize both educational efficacy and student engagement in VR settings.

MasterMinds Education VR-Agents

Left: Standard classroom setting, illustrating non-content matched environment and agent outfit. 

Right: Space-themed VR environment and agent outfit, matching the educational content on astronomy. This highlights the role of thematic consistency in VR learning engagement.

How can we support the educational system with interactive avatars?

Scientific publications and conferences

2023

  • Dai, L., Kritskaia, V., van der Velden, E., Vervoort, R., Blankendaal, M., Jung, M. M., Postma, M., & Louwerse, M. M. (2023). Text-to-Speech and Virtual Reality Agents in Primary School Classroom Environments. [Manuscript submitted for publication]. Department of Cognitive Science & Artificial Intelligence, Tilburg University.
  • Dai, L., Kritskaia, V., van der Velden, E., Jung, M. M., Postma, M., & Louwerse, M. M. (2023). Evaluating the usage of Text-To-Speech in K12 education. In Proceedings of the 2022 6th International Conference on Education and E-Learning (pp. 182–188). Association for Computing Machinery. https://doi.org/10.1145/3578837.3578864.  

2022

  • Dai, L., Jung, M. M., Postma, M., & Louwerse, M. M. (2022). A systematic review of pedagogical agent research: Similarities, differences, and unexplored aspects. Computers & Education, 190, Article 104607. https://doi.org/10.1016/j.compedu.2022.104607.  

2021

  • Dai, L. (2021). Photogrammetry-based VR interactive pedagogical agent for K12 education. In Proceedings of the 2021 International Conference on Multimodal Interaction (pp. 793–796). Association for Computing Machinery. https://doi.org/10.1145/3462244.3481280.