How do pupils experience Technology-Based Assessments? Implications for methodological approaches to measuring the User Experience based on two case studies in France and Luxembourg

Technology-based assessments (TBAs) are widely used in the education field to examine whether the learning goals were achieved. To design fair and child-friendly TBAs that enable pupils to perform at their best (i.a. independent of individual differences in computer literacy), we must ensure reliable and valid data collection. By reducing Human-Computer Interaction issues, we provide the best possible assessment conditions and user experience (UX) with the TBA and reduce educational inequalities. Good UX is thus a prerequisite for better data validity. Building on a recent case study, we investigated how pupils perform TBAs in real-life settings. We addressed the context-dependent factors resulting from the observations that ultimately influence the UX. The first case study was conducted with pupils age 6 to 7 in three elementary schools in France (n=61) in collaboration with la direction de l’évaluation, de la prospective et de la performance (DEPP). The second case study was done with pupils age 12 to 16 in four secondary schools in Luxembourg (n=104) in collaboration with the Luxembourg Centre for Educational Testing (LUCET). This exploratory study focused on the collection of various qualitative datasets to identify factors that influence the interaction with the TBA. We also discuss the importance of teachers’ moderation style and mere system-related characteristics, such as audio protocols of the assessment data. This study contribution comprises design recommendations and implications for methodological approaches to measuring pupils’ user experience during TBAs.