Abstract:
Collaborative inquiry is a widely used learning approach in higher education, where students work together to solve problems and build knowledge through sustained communication. However, as class sizes grow, instructors face challenges in efficiently assessing collaborative inquiries and providing timely, effective feedback due to the large volume of discourse data generated. This project aims to address these issues by introducing an AI-powered analytical module designed to assist instructors in formative assessment and feedback. Grounded in the CoI framework, the module automatically processes discourse data, models student inquiry patterns, and generates visualizations and explanations to support instructors’ decision-making. Throughout the project, the module will directly benefit 200 to 300 undergraduate and graduate students enrolled in inquiry-based courses. Additionally, 30 to 50 staff members will gain insights through workshops and seminars. Quasi-experiments will explore the impact of AI-assisted assessment on students’ inquiry patterns and perceived feedback effectiveness. In the long term, this project aims to extend the module’s use across all inquiry-based courses at EdUHK and beyond, enhancing problem-solving and critical thinking skills while incorporating AI into teaching and learning processes.
Code:
T0284
Principal Project Supervisors:
Keywords provided by authors:
- Collaborative inquiry
- Artificial Intelligence
- Learning analytics
- Assessment and feedback
- Higher-order thinking
Start Date:
15 Apr 2024
End Date:
14 Jun 2025
Status:
Ongoing
Result:
The project has made substantial progress, successfully gathering approximately 7000 messages from collaborative inquiry activities. These messages were coded by three trained researchers using the CoI framework, which classifies interactions into cognitive, social, and teaching presences. The coding process was rigorously evaluated, achieving an IRR score of 0.80, measured using Krippendorff’s alpha. This score indicates a high level of agreement among coders and meets the accepted threshold for reliability in qualitative research, ensuring that the data labeling is consistent and methodologically sound. This achievement fulfills one of the project’s primary objectives: ensuring reliable date for generating meaningful insights into collaborative inquiry processes.
In addition to the data analysis, the project has produced three academic outputs. Two journal articles are currently under review: one focuses on the design and effectiveness of the analytical module (Educational Psychology), and the other presents a partial evaluation of the module (Computers & Education). Furthermore, a conference paper sharing preliminary findings, particularly on AI’s role in supporting inquiry-based discussions, has been accepted for presentation at the Sixth International Conference on Quantitative Ethnography. These deliverables demonstrate the project’s ongoing contribution to the field.
Impact:
The potential impact of this project lies in the development and implementation of an AI-powered analytical module designed to support formative assessment and feedback in collaborative inquiry environments. Once developed, this module will address a significant challenge in higher education: the difficulty instructors face in efficiently managing and assessing large volumes of student discourse in increasingly larger classes. By automating the analysis of inquiry-based discussions using the CoI framework, the module will enable instructors to quickly identify students’ inquiry patterns and provide timely, personalized feedback. This innovation is expected to enhance students’ engagement critical thinking, and problem-solving skills by making the feedback process more efficient and targeted.
Although the module is still under development, the project has already begun to make an impact through its ongoing research. The creation of a comprehensive data library provides a robust foundation for both the module’s development and future research. The two journal articles currently under review will disseminate key insights about the module’s design and expected effectiveness. Additionally, a conference paper presented at the Sixth International Conference on Quantitative Ethnography has initiated discussion on AI’s role in education, fostering further exploration of AI-assisted tools in higher education.
Financial Year:
2023-24
Type:
TDG