European Public School Report 2023: Preliminary results on student population, educational trajectories, mathematics achievement, and stakeholder perceptions

Report (2023)

• Luxembourg is a highly diverse country in terms of the socioeconomic, sociocultural, and linguistic composition of its population. This diversity is reflected in the national education system with an ... [more ▼]

• Luxembourg is a highly diverse country in terms of the socioeconomic, sociocultural, and linguistic composition of its population. This diversity is reflected in the national education system with an increasing share of students speaking a language other than Luxembourgish and/or German at home. In order to deal more adequately with the increasing language diversity of the student population and to counter educational inequalities that presumably result (at least in part) from a curriculum that places high language expectations on a growing number of students, the Luxembourgish government has broadened the educational offer by introducing European public schools (EPS). These schools follow the European curriculum and allow students to select one main language of instruction among the offered language sections (i.e., German, French, and English). • By combining data from different sources (e.g., administrative student data, expert interviews with stakeholders, achievement scores in mathematics from the Luxembourg School Monitoring Programme “Épreuves Standardisées” - ÉpStan), the present report offers preliminary results on EPS in Luxembourg. They consist of (1) the societal demand for EPS; (2) the composition of the student population in EPS; (3) the perception of EPS by school management teams and parents, and tangible education outcomes in the form of (4) educational trajectories; and (5) academic achievement in mathematics among EPS students compared to their peers in schools following the Luxembourgish curriculum. Described below are the key preliminary findings for each of these aspects: (1) Since 2016, a total of six EPS have opened in different locations across Luxembourg and the amount of students attending EPS has increased considerably at both primary and secondary school level. With the number of applicants surpassing the number of places currently available in EPS, it can be concluded that there appears to be high demand for EPS. (2) With students having a low socioeconomic status (SES) and/or students speaking Portuguese at home taking up the offer of EPS less frequently than high SES students and/or students speaking French or English at home, the student population in EPS differs from the student population in schools following the Luxembourgish curriculum (e.g., nationality, language primarily spoken at home, SES). (3) School management teams and parents report a rather positive perception of EPS, with the extended linguistic offer (i.e., possibility to select a language section) being the main reason why parents select EPS for their child. (4) Looking at the educational trajectories of EPS students, preliminary results offer a tentative indication of EPS students showing less school delay than their peers in school following the Luxembourgish curriculum and high continuity in their educational trajectories (i.e., the vast majority of students remains in EPS instead of changing curriculum). (5) With regard to achievement in mathematics at primary school level, the present report indicates that students in EPS perform better than their peers in schools following the Luxembourgish curriculum. At secondary school level, EPS students perform better than their peers in Enseignement secondaire général - voie d'orientation (ESG) and in Enseignement secondaire général - voie de préparation (ESG-VP), while staying below the performance of Enseignement secondaire classique (ESC) students. Although low SES students or Portuguese speaking students in EPS show better achievement scores than their respective peers in schools following the Luxembourgish curriculum it is not yet possible to draw strong conclusions based on these preliminary findings as these student groups currently take up the EPS offer less frequently than their peers considered as advantaged in the context of schooling. Their number is currently too small to allow more robust and in-depth statistical analyses. • The present report’s findings, especially regarding the tangible educational student outcomes, however, must be considered as tentative due to important methodological limitations. Indeed, the small numbers of students in EPS, particularly so for student groups with specific background characteristics (e.g., low SES students, Portuguese speaking students), do not allow separate analyses based on language section, for example. Thus any identified pattern could be sensitive to the inclusion or exclusion of outliers (e.g., students with particularly high or low ÉpStan scores). In addition, the comprehensive EPS school system at secondary school level (i.e., common track) is compared to the ability-based tracked school system of schools following the Luxembourgish curriculum, which limits the interpretability of secondary school data. Regarding the academic achievement tasks in mathematics, it should be noted that they were developed using education standards of the Luxembourgish curriculum. It is thus possible that achievement was underestimated for EPS students (e.g., assessment of mathematical concepts that have not yet been introduced in EPS). To this date, the ÉpStan administered in EPS only assessed academic achievement in mathematics for which a bigger overlap between curricula is assumed than for language subjects (e.g., German, French). Current psychometric shortcomings (e.g., different timepoints of language introduction within the language section in EPS, task development, comparability of tasks) do not yet allow to assess academic achievement in language subjects. • Considering that the ÉpStan do not currently include a measure that operationalises the learning environment, the present report is unable to draw any conclusions regarding which EPS aspect contributes decisively in explaining the observed differences in educational outcomes. Nevertheless, three potential explanations are presented for further exploration: better linguistic fit in EPS (i.e., students learning to read and write in their native or a related language), structural differences between school offers (e.g., primary and secondary education within one institution, the institutionalized quality assurance and flexibility in teacher recruitment in EPS), and the differences in the composition of the student population (i.e., lower uptake rate of the EPS offer by low SES students and Portuguese speaking students). • The finding that low SES students and Portuguese speaking students take up the EPS offer less frequently than their high SES peers and French or English speaking students, and that the EPS student population differs from the student population in schools following the Luxembourgish curriculum, could potentially result out of three main hurdles: namely (1) the application of selection criteria considering that the demand for EPS is surpassing the number of available places (i.e., the linguistic and/or academic profile of applying students is taken into consideration); (2) lacking system knowledge regarding the characteristics of Luxembourg’s education system among all actors involved in education (which makes it difficult to take informed decisions on a student’s education); and (3) potential organizational challenges that hamper the uptake of the EPS offer (e.g., geographical location of the EPS). • In light of the tentative result that students in EPS show better educational outcomes than many of their peers in schools following the Luxembourgish curriculum, two main implications for educational policy can be deduced. First, the student composition of EPS could be diversified in a targeted manner. This could be achieved, for example, by a) encouraging EPS to target student groups considered as disadvantaged in the context of schooling (e.g., low SES students) more effectively, and by b) fostering an encompassing system knowledge (e.g., characteristics, similarities and differences of the two school offers) among all actors involved in education (e.g., teachers, parents, educational advisors, school psychologists) to allow parents to take an informed decision on their child’s education. A second implication would be to introduce certain characteristics of EPS in schools following the Luxembourgish curriculum (e.g., extending the linguistic offer as in the French literacy acquisition pilot project currently implemented in four C2.1 classes). • By progressively integrating EPS into the well-established Luxembourg School Monitoring Programme, the ÉpStan will allow for a more in-depth analysis of potential educational outcome differences between EPS and schools following the Luxembourgish in the future. With the aim of providing reliable data for evidence-based policy making in the field of education, the results from the ÉpStan could in turn be used for the creation of school offers in which all students can make use of their full academic potential irrespective of their individual background characteristics (e.g., SES, language background). [less ▲]

Educational Trajectories in Luxembourg's European Public Schools

in LUCET, SCRIPT (Ed.) European Public School Report 2023: Preliminary Results on Student Population, Educational Trajectories, Mathematics Achievement, and Stakeholder Perceptions (2023)

European Public Schools in Luxembourg: History, Overview, Attendance Rates, and Composition of the Student Population

in LUCET, SCRIPT (Ed.) European Public School Report 2023: Preliminary Results on Student Population, Educational Trajectories, Mathematics Achievement, and Stakeholder Perceptions (2023)

The impact of cognitive characteristics and image-based semantic embeddings on item difficulty

Scientific Conference (2023, April 14)

Today’s educational field has a tremendous hunger for valid and psychometrically sound items to reliably track and model students’ learning processes. Educational large-scale assessments, formative ... [more ▼]

Today’s educational field has a tremendous hunger for valid and psychometrically sound items to reliably track and model students’ learning processes. Educational large-scale assessments, formative classroom assessment, and lately, digital learning platforms require a constant stream of high-quality, and unbiased items. However, traditional development of test items ties up a significant amount of time from subject matter experts, pedagogues and psychometricians and might not be suited anymore to nowadays demands. Salvation is sought in automatic item generation (AIG) which provides the possibility of generating multiple items within a short period of time based on the development of cognitively sound item templates by using algorithms (Gierl, Lay & Tanygin, 2021). Using images or other pictorial elements in math assessment – e.g. TIMSS (Trends in International Mathematics and Science (TIMSS, Mullis et al 2009) and Programme for International Student Assessment (PISA, OECD 2013) – is a prominent way to present mathematical tasks. Research on using images in text items show ambiguous results depending on their function and perception (Hoogland et al., 2018; Lindner et al. 2018; Lindner 2020). Thus, despite the high importance, effects of image-based semantic embeddings and their potential interplay with cognitive characteristics of items are hardly studied. The use of image-based semantic embeddings instead of mainly text-based items will increase though, especially in contexts with highly heterogeneous student language backgrounds. The present study psychometrically analyses cognitive item models that were developed by a team of national subject matter experts and psychometricians and then used for algorithmically producing items for the mathematical domain of numbers & operations for Grades 1, 3, and 5 of the Luxembourgish school system. Each item model was administered in 6 experimentally varied versions to investigate the impact of a) the context the mathematical problem was presented in, and b) problem characteristics which cognitive psychology identified to influence the problem solving process. Based on samples from Grade 1 (n = 5963), Grade 3 (n = 5527), and Grade 5 (n = 5291) collected within the annual Épreuves standardisées, this design allows for evaluating whether psychometric characteristics of produced items per model are a) stable, b) can be predicted by problem characteristics, and c) are unbiased towards subgroups of students (known to be disadvantaged in the Luxembourgish school system). The developed cognitive models worked flawlessly as base for generating item instances. Out of 348 generated items, all passed ÉpStan quality criteria which correspond to standard IRT quality criteria (rit > .25; outfit >1.2). All 24 cognitive models could be fully identified either by cognitive aspects alone, or a mixture of cognitive aspects and semantic embeddings. One model could be fully described by different embeddings used. Approximately half of the cognitive models could fully explain all generated and administered items from these models, i.e. no outliers were identified. This remained constant over all grades. With the exemption of one cognitive model, we could identify those cognitive factors that determined item difficulty. These factors included well known aspects, such as, inverse ordering, tie or order effects in additions, number range, odd or even numbers, borrowing/ carry over effects or number of elements to be added. Especially in Grade 1, the chosen semantic embedding the problem was presented in impacted item difficulty in most models (80%). This clearly decreased in Grades 3, and 5 pointing to older students’ higher ability to focus on the content of mathematical problems. Each identified factor was analyzed in terms of subgroup differences and about half of the models were affected by such effects. Gender had the most impact, followed by self-concept and socioeconomic status. Interestingly those differences were mostly found for cognitive factors (23) and less for factors related to the embedding (6). In sum, results are truly promising and show that item development based on cognitive models not only provides the opportunity to apply automatic item generation but to also create item pools with at least approximately known item difficulty. Thus, the majority of developed cognitive models in this study could be used to generate a huge number of items (> 10.000.000) for the domain of numbers & operations without the need for expensive field-trials. A necessary precondition for this is the consideration of the semantic embedding the problems are presented in, especially in lower Grades. It also has to be stated that modeling in Grade 1 was more challenging due to unforeseen interactions and transfer effects between items. We will end our presentation by discussing lessons learned from models where prediction was less successful and highlighting differences between the Grades. [less ▲]

Subjektives Wohlbefinden und akademische Leistung in der Grundschule: Längsschnittliche Befunde von der 3. bis zum 5. Klasse.

Scientific Conference (2023, February 28)

How sensitive are the evaluations of a school's effectiveness to the selection of covariates in the applied value‑added model?

in Educational Assessment, Evaluation and Accountability (2022)

There is no final consensus regarding which covariates should be used (in addition to prior achievement) when estimating value-added (VA) scores to evaluate a school’s effectiveness. Therefore, we ... [more ▼]

There is no final consensus regarding which covariates should be used (in addition to prior achievement) when estimating value-added (VA) scores to evaluate a school’s effectiveness. Therefore, we examined the sensitivity of evaluations of schools’ effectiveness in math and language achievement to covariate selection in the applied VA model. Four covariate sets were systematically combined, including prior achievement from the same or different domain, sociodemographic and sociocultural background characteristics, and domain-specific achievement motivation. School VA scores were estimated using longitudinal data from the Luxembourg School Monitoring Programme with some 3600 students attending 153 primary schools in Grades 1 and 3. VA scores varied considerably, despite high correlations between VA scores based on the different sets of covariates (.66 < r < 1.00). The explained variance and consistency of school VA scores substantially improved when including prior math and prior language achievement in VA models for math and prior language achievement with sociodemographic and sociocultural background characteristics in VA models for language. These findings suggest that prior achievement in the same subject, the most commonly used covariate to date, may be insufficient to control for between-school differences in student intake when estimating school VA scores. We thus recommend using VA models with caution and applying VA scores for informative purposes rather than as a mean to base accountability decisions upon. [less ▲]

Academic Profile Development: An Investigation of Differentiation Processes Based on Students' Achievement and Grade Level

Scientific Conference (2022, April 21)

Academic achievement profiles affect students’ further development, i.e., by informing educational and professional choices. However, there is a lack of knowledge on the mechanisms behind the development ... [more ▼]

Academic achievement profiles affect students’ further development, i.e., by informing educational and professional choices. However, there is a lack of knowledge on the mechanisms behind the development of academic profiles. For research on cognitive ability profiles, specifically differentiation processes, statistical tools have been developed. In the present article, we transfer these methods for differentiation research to academic achievement data. We examine differentiation depending on students’ general level of achievement and grade level in a large Luxembourgish student sample. Students’ achievements in German, French, and Math were assessed within the Luxembourg school monitoring program. We found more balanced academic profiles with increasing achievement level. We further found more balanced profiles with increasing grade level and a positive interaction effect. [less ▲]

Need for Cognition and its relation to academic achievement in different learning environments

in Learning and Individual Differences (2022), 93

The present study investigates how Need for Cognition (NFC), an individual's tendency to engage in and enjoy thinking, relates to academic achievement in 9th grade students (N = 3.355) attending different ... [more ▼]

The present study investigates how Need for Cognition (NFC), an individual's tendency to engage in and enjoy thinking, relates to academic achievement in 9th grade students (N = 3.355) attending different school tracks to understand whether school track moderates this relation when controlling for student background variables. Using structural regression analyses, our findings revealed small and significant positive relations between NFC and academic achievement in German, French and Math. Relations were strongest in the highest and weakest in the lowest track. No significant track difference between the highest and the intermediary track could be identified; significant differences of small effect size between the intermediary and the lowest track were found in favor of the intermediary track in the relation between NFC and academic achievement in German and Math. These findings underpin the importance of NFC in academic settings, while highlighting that the relation between NFC and achievement varies with the characteristics of different learning environments. [less ▲]

Academic Achievement and Subjective Well-being: A Representative Cross-sectional Study

in Heinen, Andreas; Samuel, Robin; Vögele, Claus (Eds.) et al Wohlbefinden und Gesundheit im Jugendalter (2022)

Formal education is a very important, time-intensive, and highly consequential aspect of adolescents’ everyday life. School as well as education can influence adolescents’ well-being in both the short ... [more ▼]

Formal education is a very important, time-intensive, and highly consequential aspect of adolescents’ everyday life. School as well as education can influence adolescents’ well-being in both the short- and long-term. In return, adolescents’ well-being in- and outside school may affect their educational achievement. The objective of the present study is to investigate how self-reported dimensions of adolescents’ subjective well-being (SWB) in an educational context (i.e., academic self-concept, school anxiety, social and emotional inclusion) relate to educational pathways (regular vs. irregular school transitions; attendance of more vs. less prestigious secondary school tracks) and standardized assessment scores in key academic areas (i.e., mathematics and languages). Drawing on representative data emerging from the Luxembourg School Monitoring Programme “Épreuves Standardisées” (academic year 2018/2019), the relationship between academic achievement and students’ self-reported well-being was analysed cross-sectionally for the entire student cohorts of 5th and 9th graders. Result indicated that grades and educational pathways affect SWB, whereby in general lower ratings of SWB were observed in older students, students that experienced grade retention and students in lower secondary school tracks. Furthermore, ratings of SWB explained a significant proportion of variance in academic achievement in bot Grade 5 and Grade 9. These findings highlight the importance of student´ SWB in education. SWB may not only affect academic achievement, but also impact motivation and engagement and hence long-term educational success. Implications of the findings for research and educational debate are discussed. [less ▲]

Nouveaux résultats longitudinaux issus du monitoring scolaire national ÉpStan en première et troisième année scolaire (cycles 2.1 et 3.1) : tendance négative au niveau du développement des compétences et redoublements inefficaces

in LUCET; SCRIPT (Eds.) Rapport national sur l’éducation au Luxembourg 2021 (2021)