Fairness as seen by students - a differentiated look at perceived assessment fairness by 7th and 9th graders in Luxembourg

Scientific Conference (2022, November 10)

Assessment is probably the central factor in every educational biography: On the one hand, through direct consequences for school career decisions, on the other hand, through repercussions on each ... [more ▼]

Assessment is probably the central factor in every educational biography: On the one hand, through direct consequences for school career decisions, on the other hand, through repercussions on each student’s self-concept in the respective subject, for one's own work behavior and the perception of institutional fairness in general. A crucial factor is the subjective, perceived fairness of assessment, which has been shown to influence students' satisfaction, motivation, and attitudes toward learning (Chory-Assad, 2002; Wendorf & Alexander, 2005). The current study examines how Luxembourgish students experience fairness of assessment on the basis of representative samples of the 7iéme (N > 700 students) and 9iéme/ 5iéme (N > 2200, 35% of the total cohort) and gives a first insight into the connection with school interest and self-concept. Special attention is given to the heterogeneity of the Luxembourgish student population: the extent to which language background, socioeconomic status, and gender are related to these perceptions of fairness will be analyzed. Data was collected as part of the nationwide Épreuves standardisées in fall 2021 using the Fairness Barometer (Sonnleitner & Kovacs, 2020) - a standardized instrument to measure informational and procedural fairness in student assessment. The analyses are theoretically based on Classroom Justice Theory and educational psychology (Chory-Assad and Paulsel, 2004; Chory, 2007; Duplaga & Astani, 2010) and utilize latent variable models (SEM) to study the complex interplay between perceived assessment practices and students’ school-related motivational factors. The insights offered by this study are internationally unique in their scope and provide a first glimpse on fairness perceptions of groups of Luxembourgish students in known disadvantaged situations. Results aim to sensitize especially active teachers and educators to the central importance of assessment in schools and offer some concrete advice how to improve it. References: Chory, R. M. (2007). Enhancing student perceptions of fairness: the relationship between instructor credibility and classroom justice. Commun. Educ. 56, 89–105. doi: 10.1080/03634520600994300 Chory-Assad, R. M., and Paulsel, M. L. (2004). Classroom justice: student aggression and resistance as reactions to perceived unfairness. Commun. Educ. 53, 253–273. doi: 10.1080/0363452042000265189 Chory-Assad, R. M. (2002). Classroom justice: perceptions of fairness as a predictor of student motivation, learning, and aggression. Commun. Q. 50, 58–77. doi: 10.1080/01463370209385646 Duplaga, E. A., and Astani, M. (2010). An exploratory study of student perceptions of which classroom policies are fairest. Decision Sci. J. Innov. Educ. 8, 9–33. doi: 10.1111/j.1540-4609.2009.00241.x Sonnleitner, P., & Kovacs, C. (2020, February). Differences between students’ and teachers’ fairness perceptions: Exploring the potential of a self-administered questionnaire to improve teachers’ assessment practices. In Frontiers in Education (Vol. 5, p. 17). Frontiers Media SA. Wendorf, C. A., and Alexander, S. (2005). The influence of individual- and class-level fairness-related perceptions on student satisfaction. Contemp. Educ. Psychol. 30, 190–206. doi: 10.1016/j.cedpsych.2004.07.003 [less ▲]

Using Automatic Item Generation in the context of the Épreuves Standardisées (Épstan): A pilot study on effects of altering item characteristics and semantic embeddings

Scientific Conference (2021, November 11)

Assessing mathematical skills in national school monitoring programs such as the Luxembourgish Épreuves Standardisées (ÉpStan) creates a constant demand of developing high-quality items that is both ... [more ▼]

Assessing mathematical skills in national school monitoring programs such as the Luxembourgish Épreuves Standardisées (ÉpStan) creates a constant demand of developing high-quality items that is both expensive and time-consuming. One approach to provide high-quality items in a more efficient way is Automatic Item Generation (AIG, Gierl, 2013). Instead of creating single items, cognitive item models form the base for an algorithmic generation of a large number of new items with supposedly identical item characteristics. The stability of item characteristics is questionable, however, when different semantic embeddings are used to present the mathematical problems (Dewolf, Van Dooren, & Verschaffel, 2017, Hoogland, et al., 2018). Given culture-specific knowledge differences in students, it is not guaranteed that illustrations showing everyday activities do not differentially impact item difficulty (Martin, et al., 2012). Moreover, the prediction of empirical item difficulties based on theoretical rationales has proved to be difficult (Leighton & Gierl, 2011). This paper presents a first attempt to better understand the impact of (a) different semantic embeddings, and (b) problem-related variations on mathematics items in grades 1 (n = 2338), 3 (n = 3835) and 5 (n = 3377) within the context of ÉpStan. In total, 30 mathematical problems were presented in up to 4 different versions, either using different but equally plausible semantic contexts or altering the problem’s content characteristics. Preliminary results of IRT-scaling and DIF-analysis reveal substantial effects of both, the embedding, as well as the problem characteristics on general item difficulties as well as on subgroup level. Further results and implications for developing mathematic items, and specifically, for using AIG in the course of Épstan will be discussed. [less ▲]