Abstract:
Thanks to their well-documented benefits, team projects are increasingly popular as a valuable tool in higher education. Fully realizing such benefits, however, is predicated on the presumption of no free riding. Hence, a suitably designed assessment method for team projects should ex ante discourage free riding before project commencement and ex post punish free riders when such behavior is found to exist upon project completion. The Online Assessment System for Individual Scores (OASIS) has been implemented by teachers across seven universities in the U.S., Hong Kong, mainland China and India. It is simpler to use than the popular online assessment system Comprehensive Assessment for Team-Member Effectiveness (CATME) that employs a statistical analysis of student survey data. Compared to another popular system, Web Peer Assessment (WebPA), it is less vulnerable to a team member’s strategic behavior that can bias individual scores. Highlighting the process’ practicality and usefulness are the continued interest in using OASIS of all participating teachers for their future courses, and student surveys indicating that OASIS can improve student perception on the educational value of team projects. Hence, OASIS deserves serious consideration by teachers worldwide who are already or consider using team projects in their teaching courses.
Code:
T0198
Principal Project Supervisors:
Keywords provided by authors:
Start Date:
01 Feb 2018
End Date:
31 Jul 2019
Status:
Completed
Result:
OASIS has benefited 28 teachers and ~1,800 students at nine universities in Hong Kong, China, India and the U.S. Based on the survey data collected from the students and teachers, our key findings are as follows. First, OASIS encourages student participation in a team project through its contractual commitment and performance-based scoring method. Second, OASIS identifies and quantifies the extent of free ridership via negotiation in an end-of-project meeting among team members, thereby generating mutually agreed peer assessment data for individual scoring. Third, OASIS uses the peer assessment data to determine a team member’s relative contribution based on median estimation, which is less susceptible to gaming and outlier biases than the alternative of mean estimation. Finally, OASIS automatically assigns individual scores to members of a project team based on their estimated relative contributions and the project’s overall score set by a teacher, yielding individual scores that obey the principles of fairness and due diligence in academic assessment. Based on the current users’ positive feedback, we expect that an increasing number of teachers and students will use OASIS in the coming academic years.
Impact:
OASIS has been used by 28 teachers and ~1,800 students at nine universities in Hong Kong, China, India and the U.S. The empirical evidence documented by two refereed publications affirms that (1) OASIS encourages student participation in team projects; (2) it can detect free riding in a team project; (3) it fairly scores individual contributions by members of a team project; (4) it is user friendly; and (5) it benefits students, teachers and university management. It will benefit another ~1,000 students from 4 universities of these regions in the future. Yunnan Normal University will implement OASIS in the 2019-2020 academic year, further affirming the usefulness of the proposed new process. In addition, our newly signed commercial user (Goabroad Limited) indicates OASIS’s applicability beyond higher education institutes, thus underscoring this TDG project’s knowledge transfer to the business sector. Hence, with additional efforts in marketing and user recruitment, OASIS can become a useful peer assessment method with worldwide impact.
Deliverables:
Books/ Book Chapters/ Journal Articles/ Conference Papers
Chiu, M.M., Woo, C.K., Shiu, A., Liu, Y., & Luo, B. X. (2019). Reducing costly free-rider effects via OASIS, International Journal of Comparative Education and Development, 22(1), 30-48. https://doi.org/10.1108/IJCED-07-2019-0041 (https://repository.eduhk.hk/en/publications/reducing-costly-free-rider-e...)
Woo, C. K., Shiu, A., Chiu, M. M., & Liu, Y. (2018). Empirics of an online assessment system for individual scores (OASIS). Asian Journal of Educational Research, 6(2), 16-42. (https://repository.eduhk.hk/en/publications/empirics-of-an-online-assess...)
Seminars/ Presentations/ Sharing Sessions
Liu, L. (2018, April 12). OASIS: An Online Assessment System for Individual Scores. Paper presented at the International Young Scholar Forum, Changzhou University, China. (No. of participants: 12 teachers and Research Postgraduate (RPg) students)
Woo, C. K. & Shiu, A. (2018, April 23). OASIS: An Online Assessment System for Individual Scores. Seminar presented at The Hong Kong Polytechnic University. (No. of participants: 28 teachers).https://repository.eduhk.hk/en/publications/8fc83d63-1141-4c17-9eca-32cd814e898b
Woo, C. K., Shiu, A., & Liu, L. (2018, July 16). OASIS: An Online Assessment System for Individual Scores. Seminar presented at The Education University of Hong Kong (No. of participants: 18 teachers).https://repository.eduhk.hk/en/publications/e30acf54-1a08-4c91-ae52-0dbb29d3dc89
Shiu, A. (2018, August 29). OASIS: An Online Assessment System for Individual Scores. Seminar presented at the annual retreat of School of Accounting and finance at the Hong Kong Polytechnic University (No. of participants: 90 teachers).
胡志強和劉贇(2018.10.31)：OASIS：國際化的高校網路個人分數評估系統，發表於雲南師範大學，昆明市，中國。 (No. of participants: 60 presidents and deans of universities from China, Thailand, Vietnam, and Myanmar).(https://repository.eduhk.hk/en/publications/oasis%E5%9C%8B%E9%9A%9B%E5%8...)
Teaching and Learning Resources/ Materials (including online resources)
The OASIS website: (http://www.eduhk.hk/oasis/)https://repository.eduhk.hk/en/publications/1926b2e9-72ef-41cf-a77b-2c227fc66d03
The OASIS user manualhttps://repository.eduhk.hk/en/publications/ea16db26-59cb-4395-9fa1-3c000c39fa4e
The OASIS Excel worksheet of Statement of Output (SO) generator
The OASIS English and Chinese survey questionnaires surveys
Financial Year:
2017-18
Type:
TDG