Targeting data collection in games based assessment

Targeting data collection in games based assessment

Targeting data collection in games based assessment

By Clare Walsh and Christian Bokhove

Abstract

“Educational game performance data has the potential to allow new types of complex, procedural skills to beassessed. However, prior work has shown that gameplay data do not readily align to existing assessment validation paradigms, and game performance scores are difficult to use for proficiency testing. A new assessmentparadigm that can cope with the nature of gameplay data has not emerged. In this paper, we uncovered a range of structural issues in data collection caused by, and potentially solved by, the engineered environments ingames. Choice and the iterative nature of games were found to allow curriculum specialization. We found evidence that early attempts at new games are less reliable and perhaps best discarded, and we propose a solution toweight scores to reflect novelty in repeated tasks. We found capturing the effect of competitor or collaboratorability on performance challenging but propose the potential for bots to resolve this. Finally, we also investigatedthe use of response time as a proxy for ability. The physical measure of time proved difficult and potentiallyunfair to use, but we propose a possible stochastic treatment of speed that could allow scoring some skills in some games using response time.”

Reference

Walsh, C., & Bokhove, C. (2021, October 17). Targeting data collection in games based assessment. Retrieved February 23, 2022, from https://www.sciencedirect.com/science/article/pii/S2666557321000252

Keyword

Games, evaluation methodologies, data science applications in education, 21st century abilities, research