Assessing Student Achievement in Probability Problem Solving Using Collaboration Process Data

Development and Use of a Scoring Rubric

Edición de la obra Assessing Student Achievement in Probability Problem Solving Using Collaboration Process Data

Collaborative problem solving (CPS) is a critical competency, because much of the work people do occurs in a social context involving direct collaboration. Meanwhile, schools are being pressured to reduce the amount of time devoted to large-scale assessments, and to adopt more natural or authentic assessments. It may be possible to address both these issues at once, if collaboration experiences are viewed as opportunities to assess student achievement.However, several issues arise in evaluating individuals’ problem solving skills in a collaborative context: (1) collaborative learning outcomes may obscure individuals’ contributions, making it difficult to isolate individuals’ performance; (2) outcome-based measures may ignore the processes of individuals’ or groups’ problem solving, thus leading to inaccurate estimations of individuals’ or groups’ knowledge, abilities and skills; and (3) prior evaluative focus of CPS research has usually been on social aspects of collaboration rather than domain-relevant cognitive skills. Therefore, the present study aimed to develop a process-based scoring rubric to evaluate individual student achievement in problem solving using collaboration process data. Furthermore, this study explored how group ability composition affects group performance and individual learning gains. The content domain was solving mathematical combinatorics and probability word problems. Participants included 306 Chinese high school students, who performed the following three tasks in order: (1) an individual pretest with seven problems; (2) a collaborative task with three problems; and (3) an individual posttest with seven problems. The results were as follows: First, a four-indicator scoring rubric was developed to evaluate student achievement in solving combinatorics and probability problems using collaboration process data. Evidence suggested that the scoring rubric can be considered reliable and valid in terms of being used as an individual assessment and a teaching tool. Thus, this scoring rubric may provide insights useful for developing relevant performance assessments on more complex and authentic performance tasks. Second, significant differences were found in group performance among dyads as a function of the group’s minimum student ability. Third, students working in dyads with higher maximum or average ability tended to gain more after collaboration. Finally, certain collaborative problem solving behaviors could be linked to individual learning gains.