Skip to content
EdUHK

Learning and Teaching Initiatives funded by the UGC

VR-AI Integrated Immersive Putonghua Learning Platform: Development of Scenario-Based Teaching Materials

Abstract:

Proficiency in Putonghua is increasingly vital for academic success and the cultivation of national identity among contemporary youth. Aligning with the University’s strategic development 2026-2031, Putonghua learning could also allow our students to gain essential skills and knowledge as future educators. However, a significant and persistent gap remains between students’ theoretical language knowledge and the practical oral proficiency demanded in real-world professional settings (Zhou, 2020). This disconnection is especially acute for Cantonese-speaking learners of Putonghua, a tonal language that requires nuanced oral mastery. As announced by the Hong Kong Education Bureau (EDB, 2025), the Language Proficiency Requirement (LPR) required for Qualified Putonghua Teachers has been elevated from Putonghua Shuiping Ceshi (PSC) Level 2B to Level 2A, which poses a huge challenge for many students. This misalignment underscores an urgent need for innovative pedagogical strategies that can effectively bridge this critical gap and fully prepare graduates for linguistically demanding careers.

  • This project responds with a pioneering VR-AI integrated immersive Putonghua learning platform, set to revolutionise language education for all students at EdUHK and beyond. The platform introduces three key research-informed innovations: Authentic Scenario-Based Simulations: Learners engage in realistic role-playing activities across diverse contexts—such as academic discussions, professional meetings, classroom teaching, and formal oral examinations—enabling situated language practice that mirrors real-life communication.
  • Adaptive AI-Driven Feedback: Utilising automatic speech recognition (ASR), the platform provides real-time, personalised diagnostics on pronunciation, tone, fluency, and grammatical accuracy, fostering continuous improvement.
  • Curriculum-Aligned Teaching Materials: All content is meticulously designed to align with EdUHK’s academic standards, ensuring seamless integration into existing programmes.

What sets this initiative apart is its integration of VR with AI—while AI-assisted language learning is increasingly common, the incorporation of fully immersive VR environments into Putonghua pedagogy remains rare, positioning this project at the forefront of educational innovation.

The platform directly supports Chinese-major students in meeting the EDB’s LPR by targeting PSC Level 2A attainment, essential for teaching certification. It also extends substantial benefits to non-Chinese majors, aiming to elevate overall Putonghua proficiency and boost PSC pass rates across the university. With an anticipated reach of over 5,000 students in the next five years through integration into PGDE, BEd, BA and CLE courses, this initiative not only addresses an urgent educational need but also establishes a scalable, research-backed model for technology-enhanced language learning. By doing so, it positions EdUHK as a leader in pedagogical innovation, empowering all students to excel in settings where advanced Putonghua communication is essential.

Code:

T0300

Principal Project Supervisors:

Keywords provided by authors:

Start Date:

01 Feb 2026

End Date:

31 Jul 2027

Status:

Ongoing

Result:

We will develop an immersive Putonghua learning platform integrating Virtual Reality (VR) and Artificial Intelligence (AI). This platform will deploy at least four role-playing teaching toolkits spanning academic, professional, pedagogical, and examination contexts, featuring over twenty-five VR scenarios and more than five hundred AI chatbot dialogue variations with instantaneous feedback through ASR. Designed for seamless integration into PGDE/BEd/BA/CLE programs, the platform will be implemented within specific courses such as CHI5116 PGDE Putonghua Language Immersion, PTH1053 Foundation Putonghua, CHI3916 Chinese L2 Teaching Pedagogy I: Listening and Speaking, and CLE3228 PSC Preparation Course over the next five years to establish a sustainable adoption pathway. Key targets include enabling 90% or more of final-year Chinese major undergraduates to achieve PSC Level 2B/2A or higher. For non-Chinese major students, we target a 10% increase in oral fluency scores and a 10% reduction in pronunciation errors, measured via pre-/post-tests within classroom formative assessments. Research dissemination will include at least two academic conference publications. Additionally, we plan to train over two hundred educators through a minimum of four specialised workshops.


Impact:

The platform leverages multi-sensory VR environments, proven to reduce cognitive load in language acquisition (Barsalou, 2008, 2016, 2020), and immersive role-playing simulations to develop real-world communication strategies. This approach directly addresses the EDB’s mandate for practical competency development in teacher education. The AI dialogue system provides real-time pronunciation diagnostics, enabling personalised corrective feedback. Crucially, the system’s 500+ contextual dialogue variations specifically target tonal challenges faced by Hong Kong learners, aligning with PSC Level 2A certification requirements. These features underpin our targets for Chinese major undergraduates (90% achieving PSC 2B/2A+) and non-major students (10% oral score increase, 10% error reduction).


Furthermore, through promotional outreach and workshops led by our external member, Dr Leung Wai Ying, a serving vice-principal and chairman of a key extracurricular masters’ association, this innovative platform will be introduced to local primary and secondary schools. This direct channel ensures that the project’s benefits extend beyond the University, providing school teachers and students with access to cutting-edge resources for enhancing Putonghua teaching and learning. This initiative not only broadens the project’s impact but also fosters a stronger university-school partnership, creating a sustainable ecosystem for oral language education across different educational levels in Hong Kong.


Financial Year:

2025-26

Type:

TDG