Validierung und psychometrische Analyse automatisch generierter Mathematikaufgaben im Bereich Zahlen und Operationen

Scientific Conference (2022, March 09)

Différences de performance dans les compétences langagières et en lecture entre élèves à parcours scolaire régulier et irrégulier, issus de familles immigrées portugaises au Luxembourg.

in University of Luxembourg, LUCET; Ministère de l’Éducation nationale, de l’Enfance et de la Jeunesse, SCRIPT (Eds.) Rapport national sur l’éducation au Luxembourg 2021 (2021)

Neue längsschnittliche Befunde aus dem nationalen Bildungsmonitoring ÉpStan in der 1. und 3. Klasse: Negativer Trend in der Kompetenzentwicklung und kein Erfolg bei Klassenwiederholungen.

in LUCET; SCRIPT (Eds.) Nationaler Bildungsbericht Luxemburg 2021 (2021)

Befunde aus dem nationalen Bildungsmonitoring ÉpStan vor dem Hintergrund der COVID-19 Pandemie (Supplement)

in LUCET; SCRIPT (Eds.) Nationaler Bildungsbericht Luxemburg 2021 (2021)

Résultats du monitoring scolaire national ÉpStan dans le contexte de la pandémie de COVID-19

in LUCET; SCRIPT (Eds.) Rapport national sur l’éducation au Luxembourg 2021 (2021)

Using Automatic Item Generation in the context of the Épreuves Standardisées (Épstan): A pilot study on effects of altering item characteristics and semantic embeddings

Scientific Conference (2021, November 11)

Assessing mathematical skills in national school monitoring programs such as the Luxembourgish Épreuves Standardisées (ÉpStan) creates a constant demand of developing high-quality items that is both ... [more ▼]

Assessing mathematical skills in national school monitoring programs such as the Luxembourgish Épreuves Standardisées (ÉpStan) creates a constant demand of developing high-quality items that is both expensive and time-consuming. One approach to provide high-quality items in a more efficient way is Automatic Item Generation (AIG, Gierl, 2013). Instead of creating single items, cognitive item models form the base for an algorithmic generation of a large number of new items with supposedly identical item characteristics. The stability of item characteristics is questionable, however, when different semantic embeddings are used to present the mathematical problems (Dewolf, Van Dooren, & Verschaffel, 2017, Hoogland, et al., 2018). Given culture-specific knowledge differences in students, it is not guaranteed that illustrations showing everyday activities do not differentially impact item difficulty (Martin, et al., 2012). Moreover, the prediction of empirical item difficulties based on theoretical rationales has proved to be difficult (Leighton & Gierl, 2011). This paper presents a first attempt to better understand the impact of (a) different semantic embeddings, and (b) problem-related variations on mathematics items in grades 1 (n = 2338), 3 (n = 3835) and 5 (n = 3377) within the context of ÉpStan. In total, 30 mathematical problems were presented in up to 4 different versions, either using different but equally plausible semantic contexts or altering the problem’s content characteristics. Preliminary results of IRT-scaling and DIF-analysis reveal substantial effects of both, the embedding, as well as the problem characteristics on general item difficulties as well as on subgroup level. Further results and implications for developing mathematic items, and specifically, for using AIG in the course of Épstan will be discussed. [less ▲]

The factor structure of mathematical abilities in Luxembourg’s national school monitoring: Its stability over elementary school and relations to, gender, language background, and SES

Scientific Conference (2021, July)

Mathematics skills are the fundament of modern societies, especially those based on a knowledge-economy. The age of digitalization renders mathematics education even more crucial since it builds the ... [more ▼]

Mathematics skills are the fundament of modern societies, especially those based on a knowledge-economy. The age of digitalization renders mathematics education even more crucial since it builds the starting point for all STEM-related fields. Consequently, mathematics is at the core of numerous educational Large-Scale Assessments on international (e.g. PISA, TIMSS) or national level (e.g. NAEP, NEPS, SNSA). Although the underlying test development frameworks are most often multi-dimensional or hierarchical, psychometric analyses usually focus on a single latent factor that represents a rather vague general mathematical ability. How and to what extent this simplification affects educational studies that rely on these data remains unclear. The present study takes Luxembourg’s national school monitoring program ÉpStan as example to tackle this question and clarify the consequences. ÉpStan’s mathematics test is conducted annually in elementary school Grades 1, 3, and 5 and is comprised of around 50 to 70 items. Since ÉpStan captures competencies of all students biyearly, each analysis will be based on the full cohort (n > 5000). First, we will investigate whether the curriculum-based test framework for mathematics can psychometrically be represented in a related (multi-dimensional) confirmatory factor model including the domains numbers & operations and space & form. This will be done in Grades 1, 3, and 5. Second, we will study the factor model’s cross-sectional stability within each Grade (over three consecutive years) and longitudinal stability between Grades. Finally, we will study the factors’ relations to students’ cognitive and sociodemographic characteristics and compare the results with correlations found using the most widely used one-dimensional model of mathematical abilities. Based on the results, we will discuss implications not only for educational studies that often uncritically make use of large-scale assessment data, but also highlight the consequences for group-level feedback that is based on such assessments. [less ▲]

Long-term Relevance and Interrelation of Symbolic and Non-symbolic Abilities in Mathematical-numerical Development: Evidence from Large-scale Assessment Data

in Cognitive Development (2021), 58

LEARN Newsletter - Édition 2021

Book published by LEARN (2021)

Finger Rapid Automatized Naming (RAN) predicts the development of numerical representations better than finger gnosis

in Cognitive Development (2020), 53