Early Childhood Education and Care in Luxembourg and Language: Performance Closing the Gap between different Language Backgrounds?

Scientific Conference (2023, August 21)

Achievement gaps between students of different family backgrounds have been found in many countries (e.g. Stanat & Christensen, 2006). They are not only based on socioeconomic status or immigration ... [more ▼]

Achievement gaps between students of different family backgrounds have been found in many countries (e.g. Stanat & Christensen, 2006). They are not only based on socioeconomic status or immigration background, but also on home language: If children do not speak the language of instruction at home, they are often disadvantaged in school and perform worse in school performance tests than students speaking the instruction language at home (e.g. Van Staden et al., 2016). Low SES increases the risk that children with an L2 instruction language are disadvantaged (Cummins, 2018). With rising numbers of global migration (Edmond, 2020), these disparities in educational systems can be expected to become more distinct in the future. Luxembourg is a trilingual country with an already highly diverse student population in terms of nationality and language background, with 67 % of elementary school students not speaking the first instruction language Luxembourgish at home (MENJE & SCRIPT, 2022). It is therefore a prime example to study these educational challenges ahead of time. In addition to the “super-diversity” of Luxembourg, students of different language backgrounds have to deal with a highly demanding language curriculum at school, in which the instruction language switches first from Luxembourgish to German and then to French in secondary education. In consequence, many students face challenges in acquiring language and literacy skills (e.g. Hornung et al., 2021) – leading to distinct gaps between students of different language backgrounds. One possible way to decrease such disparities might be an early and extensive participation in early childhood education and care (ECEC). Participation in ECEC, that is “any regulated arrangement that provides education and care to children from birth to compulsory primary school age” (European Commission, n.d.), has been shown to have positive effects on language development and other cognitive abilities. These effects differ between age groups. For young children from age 0 to 3, a Norwegian study found that scaling up early ECEC improved early language skills at the age of seven (Drange & Havnes, 2015). However, a review also indicated research on this age group was scarcer and produced more varied findings (Melhuish et al., 2015). For children between the ages 3 and 6, effects on language and other cognitive skills were more consistently positive (Melhuish et al., 2015). In children with differing home language backgrounds, this association was stronger than in those who spoke the majority language at home (Ansari et al., 2021). This study aims to investigate if these findings hold in the multilingual and diverse school context of Luxembourg and to analyze the effects of ECEC attendance on language performance, differentiated by the student’s home language background and the particular type of ECEC (non-formal daycare vs formal early education). Based on the presented literature, we hypothesize that (1) participation in ECEC, formal and nonformal, is associated with higher listening comprehension in Luxembourgish (i.e. the first instruction language) in grade 1, that (2) the associations are moderated by the children home language background where greater associations are expected for children who do not speak the instruction language at home and that (3) participation in formal ECEC explains more variance than participation in nonformal ECEC. Methodology, Methods, Research Instruments or Sources Used To answer our research questions, we draw on a large-scale dataset of n = 5.952 first graders from the Luxemburg school monitoring programme ÉpStan (Épreuves Standardisées) in 2021. The ÉpStan includes questionnaires and written competence tests in key school areas that are implemented every year for all Luxembourgish students in grades 1, 3, 5, 7, and 9. Its aim is a.o. to objectively assess the long-term performance of the Luxembourgish school system. For our study, we focus on Luxembourg listening comprehension in grade 1, which is assessed with different text formats, such as dialogues, short stories or radio broadcasts presented on CDs. The test is measuring different sub-skills, defined by the national curriculum, such as understanding one’s interlocutor, locating, understanding and interpreting information, and applying listening strategies (recognition of noises and voices). Information on ECEC participation is assessed retrospectively in parent questionnaires for crèches (non-formal ECEC targeted at 0-4 year olds) and for précoce (formal ECEC, targeted at 3 year olds). Home language background is assessed by self-report in the student questionnaire and categorised into five groups: a) Luxembourgish, b) French, c) Portuguese, d) bilingual Luxembourgish / French and e) bilingual Luxembourgish / Portuguese. After checking whether the prerequisites for the analyses are met, we calculate a multivariate regression model with the two ECEC types as binary predictors and other family background variables as control for hypothesis (1). For hypothesis (2), we test whether home language background moderates the association between ECEC and language performance by adding interaction terms of home language group with each ECEC type to our regression model. For hypothesis (3), we compare the incremental variance explained by each ECEC type. Conclusions, Expected Outcomes or Findings We expect our outcomes to show that attendance in both ECEC types have positive associations with Luxembourgish listening comprehension in first grade, in line with many findings on the topic. Additionally, attendance in formal ECEC is expected to explain more variance in Luxembourgish listening comprehension than attendance in nonformal ECEC as Luxembourgish is the main instruction language in formal ECEC. In nonformal ECEC institutions, language policies are usually less rigid and more plurilingual. We also expect significant moderations of this effect by home language background: We do not expect a strong effect of both formal and nonformal ECEC on listening comprehension for children who speak only Luxembourgish at home, as they are expected to have developed these skills at home. Children who do not speak Luxembourgish at home are, on the other hand, expected to benefit more from ECEC attendance. This would then indicate that more time spent in ECEC institutions fostered their basic skills in the instruction language and helped gain better listening performance. Being competent in the instruction language is essential for further learning. Without the language skills, children are unable to connect to the school’s input (Schleppegrell, 2001). All in all, the findings might help to understand the effects of two different ECEC types in Luxembourg for children of different language backgrounds – indicating for whom ECEC attendance should be explicitly encouraged. It might also give us valuable hints towards characteristics of ECEC that are especially helpful to further language skills and thus, later school performance. Implications on possible policy decisions with the goal of closing achievement gaps and furthering educational equality will be discussed. References Ansari, A., Pianta, R. C., Whittaker, J. E., Vitiello, V., & Ruzek, E. (2021). Enrollment in public-prekindergarten and school readiness skills at kindergarten entry: Differential associations by home language, income, and program characteristics. Early Childhood Research Quarterly, 54, 60–71. https://doi.org/10.1016/j.ecresq.2020.07.011 Cummins, J. (2018). Urban Multilingualism and Educational Achievement: Identifying and Implementing Evidence-Based Strategies for School Improvement. In P. Van Avermaet, S. Slembrouck, K. Van Gorp, S. Sierens, & K. Maryns (Eds.), The Multilingual Edge of Education (p. 67–90). Palgrave Macmillan. https://doi.org/10.1057/978-1-137-54856-6_4 Drange, N., & Havnes, T. (2015). Child Care Before Age Two and the Development of Language and Numeracy: Evidence from a Lottery. Discussion Papers. Statistics Norway. Research Department., 808. https://doi.org/10.2139/ssrn.2582539 Edmond, C. (2020, January 10). Global migration, by the numbers. World Economic Forum. https://www.weforum.org/agenda/2020/01/iom-global-migration-report-international-migrants-2020/ European Commission. (n.d.). Early childhood education and care initiatives. Retrieved 23rd May 2022, from https://education.ec.europa.eu/node/1702 Hornung, C., Wollschläger, R., Keller, U., Esch, P., Muller, C., & Fischbach, A. (2021). Neue längsschnittliche Befunde aus dem nationalen Bildungsmonitoring ÉpStan in der 1. und 3. Klasse. Negativer Trend in der Kompetenzentwicklung und kein Erfolg bei Klassenwiederholungen. In LUCET & SCRIPT (Eds.), Nationaler Bildungsbericht Luxemburg 2021 (p. 44–55). LUCET & SCRIPT. Melhuish, E., Ereky-Stevens, K., Petrogiannis, K., Ariescu, A., Penderi, E., Rentzou, K., Tawell, A., Leseman, P., & Broekhuisen, M. (2015). A review of research on the effects of early childhood education and care (ECEC) on child development [Technical Report.]. MENJE & SCRIPT. (2022). Education system in Luxembourg. Key Figures. edustat.lu Schleppegrell, M. J. (2001). Linguistic Features of the Language of Schooling. Linguistics and Education, 12(4), 431–459. https://doi.org/10.1016/S0898-5898(01)00073-0 Stanat, P., & Christensen, G. (2006). Where Immigrant Students Succeed—A Comparative Review of Performance and Engagement in PISA 2003. https://www.oecd.org/education/school/programmeforinternationalstudentassessmentpisa/whereimmigrantstudentssucceed-acomparativereviewofperformanceandengagementinpisa2003.htm Van Staden, S., Bosker, R., & Bergbauer, A. (2016). Differences in achievement between home language and language of learning in South Africa: Evidence from prePIRLS 2011. South African Journal of Childhood Education, 6(1), 10. https://doi.org/10.4102/sajce.v6i1.441 [less ▲]

The impact of cognitive characteristics and image-based semantic embeddings on item difficulty

Scientific Conference (2023, April 14)

Today’s educational field has a tremendous hunger for valid and psychometrically sound items to reliably track and model students’ learning processes. Educational large-scale assessments, formative ... [more ▼]

Today’s educational field has a tremendous hunger for valid and psychometrically sound items to reliably track and model students’ learning processes. Educational large-scale assessments, formative classroom assessment, and lately, digital learning platforms require a constant stream of high-quality, and unbiased items. However, traditional development of test items ties up a significant amount of time from subject matter experts, pedagogues and psychometricians and might not be suited anymore to nowadays demands. Salvation is sought in automatic item generation (AIG) which provides the possibility of generating multiple items within a short period of time based on the development of cognitively sound item templates by using algorithms (Gierl, Lay & Tanygin, 2021). Using images or other pictorial elements in math assessment – e.g. TIMSS (Trends in International Mathematics and Science (TIMSS, Mullis et al 2009) and Programme for International Student Assessment (PISA, OECD 2013) – is a prominent way to present mathematical tasks. Research on using images in text items show ambiguous results depending on their function and perception (Hoogland et al., 2018; Lindner et al. 2018; Lindner 2020). Thus, despite the high importance, effects of image-based semantic embeddings and their potential interplay with cognitive characteristics of items are hardly studied. The use of image-based semantic embeddings instead of mainly text-based items will increase though, especially in contexts with highly heterogeneous student language backgrounds. The present study psychometrically analyses cognitive item models that were developed by a team of national subject matter experts and psychometricians and then used for algorithmically producing items for the mathematical domain of numbers & operations for Grades 1, 3, and 5 of the Luxembourgish school system. Each item model was administered in 6 experimentally varied versions to investigate the impact of a) the context the mathematical problem was presented in, and b) problem characteristics which cognitive psychology identified to influence the problem solving process. Based on samples from Grade 1 (n = 5963), Grade 3 (n = 5527), and Grade 5 (n = 5291) collected within the annual Épreuves standardisées, this design allows for evaluating whether psychometric characteristics of produced items per model are a) stable, b) can be predicted by problem characteristics, and c) are unbiased towards subgroups of students (known to be disadvantaged in the Luxembourgish school system). The developed cognitive models worked flawlessly as base for generating item instances. Out of 348 generated items, all passed ÉpStan quality criteria which correspond to standard IRT quality criteria (rit > .25; outfit >1.2). All 24 cognitive models could be fully identified either by cognitive aspects alone, or a mixture of cognitive aspects and semantic embeddings. One model could be fully described by different embeddings used. Approximately half of the cognitive models could fully explain all generated and administered items from these models, i.e. no outliers were identified. This remained constant over all grades. With the exemption of one cognitive model, we could identify those cognitive factors that determined item difficulty. These factors included well known aspects, such as, inverse ordering, tie or order effects in additions, number range, odd or even numbers, borrowing/ carry over effects or number of elements to be added. Especially in Grade 1, the chosen semantic embedding the problem was presented in impacted item difficulty in most models (80%). This clearly decreased in Grades 3, and 5 pointing to older students’ higher ability to focus on the content of mathematical problems. Each identified factor was analyzed in terms of subgroup differences and about half of the models were affected by such effects. Gender had the most impact, followed by self-concept and socioeconomic status. Interestingly those differences were mostly found for cognitive factors (23) and less for factors related to the embedding (6). In sum, results are truly promising and show that item development based on cognitive models not only provides the opportunity to apply automatic item generation but to also create item pools with at least approximately known item difficulty. Thus, the majority of developed cognitive models in this study could be used to generate a huge number of items (> 10.000.000) for the domain of numbers & operations without the need for expensive field-trials. A necessary precondition for this is the consideration of the semantic embedding the problems are presented in, especially in lower Grades. It also has to be stated that modeling in Grade 1 was more challenging due to unforeseen interactions and transfer effects between items. We will end our presentation by discussing lessons learned from models where prediction was less successful and highlighting differences between the Grades. [less ▲]

The crucial role of language in mathematical development

Scientific Conference (2022, November 17)

Basic mathematics skills build on nonverbal number sense But these innate non-verbal skills are insufficient to develop symbolic exact number concepts and to learn arithmetic. Language development allows ... [more ▼]

Basic mathematics skills build on nonverbal number sense But these innate non-verbal skills are insufficient to develop symbolic exact number concepts and to learn arithmetic. Language development allows the acquisition of number words and math vocabulary, crucial for developing basic exact number concepts and arithmetic skills. This presentations highlights five key aspects on how language influences mathematical development. First, language is a building block for basic math skills. Second, number naming systems affect number transcoding. Third, multilingual students calculate better in the language in which they have learned numbers. Forth, children's home language influences their mathematics achievement. And finally, the mastery of the language of instruction has a strong impact on mathematics achievement. The implication of these key aspects are discussed with regard to education and instruction in schools. [less ▲]

Validation and Psychometric Analysis of 32 cognitive item models spanning Grades 1 to 7 in the mathematical domain of numbers & operations

Scientific Conference (2022, November)

Today’s educational field has a tremendous hunger for valid and psychometrically sound items to reliably track and model students’ learning processes. Educational large-scale assessments, formative ... [more ▼]

Today’s educational field has a tremendous hunger for valid and psychometrically sound items to reliably track and model students’ learning processes. Educational large-scale assessments, formative classroom assessment, and lately, digital learning platforms require a constant stream of high-quality, and unbiased items. However, traditional development of test items ties up a significant amount of time from subject matter experts, pedagogues and psychometricians and might not be suited anymore to nowadays demands. Salvation is sought in automatic item generation (AIG) which provides the possibility of generating multiple items within a short period of time based on the development of cognitively sound item templates by using algorithms (Gierl & Haladyna, 2013; Gierl et al., 2015). The present study psychometrically analyses 35 cognitive item models that were developed by a team of national subject matter experts and psychometricians and then used for algorithmically producing items for the mathematical domain of numbers & shapes for Grades 1, 3, 5, and 7 of the Luxembourgish school system. Each item model was administered in 6 experimentally varied versions to investigate the impact of a) the context the mathematical problem was presented in, and b) problem characteristics which cognitive psychology identified to influence the problem solving process. Based on samples from Grade 1 (n = 5963), Grade 3 (n = 5527), Grade 5 (n = 5291), and Grade 7 (n = 3018) collected within the annual Épreuves standardisées, this design allows for evaluating whether psychometric characteristics of produced items per model are a) stable, b) can be predicted by problem characteristics, and c) are unbiased towards subgroups of students (known to be disadvantaged in the Luxembourgish school system). After item calibration using the 1-PL model, each cognitive model was analyzed in-depth by descriptive comparisons of resulting IRT parameters, and the estimation of manipulated problem characteristics’ impact on item difficulty by using the linear logistic test model (LLTM, Fischer, 1972). Results are truly promising and show negligible effects of different problem contexts on item difficulty and reasonably stable effects of altered problem characteristics. Thus, the majority of developed cognitive models could be used to generate a huge number of items (> 10.000.000) for the domain of numbers & operations with known psychometric properties without the need for expensive field-trials. We end with discussing lessons learned from item difficulty prediction per model and highlighting differences between the Grades. References: Fischer, G. H. (1973). The linear logistic test model as an instrument in educational research. Acta Psychologica, 36, 359-374. Gierl, M. J., & Haladyna, T. M. (Eds.). (2013). Automatic item generation: Theory and practice. New York, NY: Routledge. Gierl, M. J., Lai, H., Hogan, J., & Matovinovic, D. (2015). A Method for Generating Educational Test Items That Are Aligned to the Common Core State Standards. Journal of Applied Testing Technology, 16(1), 1–18. [less ▲]

Validierung und psychometrische Analyse automatisch generierter Mathematikaufgaben im Bereich Zahlen und Operationen

Scientific Conference (2022, March 09)

Neue längsschnittliche Befunde aus dem nationalen Bildungsmonitoring ÉpStan in der 1. und 3. Klasse: Negativer Trend in der Kompetenzentwicklung und kein Erfolg bei Klassenwiederholungen.

in LUCET; SCRIPT (Eds.) Nationaler Bildungsbericht Luxemburg 2021 (2021)

Différences de performance dans les compétences langagières et en lecture entre élèves à parcours scolaire régulier et irrégulier, issus de familles immigrées portugaises au Luxembourg.

in University of Luxembourg, LUCET; Ministère de l’Éducation nationale, de l’Enfance et de la Jeunesse, SCRIPT (Eds.) Rapport national sur l’éducation au Luxembourg 2021 (2021)

Befunde aus dem nationalen Bildungsmonitoring ÉpStan vor dem Hintergrund der COVID-19 Pandemie (Supplement)

in LUCET; SCRIPT (Eds.) Nationaler Bildungsbericht Luxemburg 2021 (2021)

Résultats du monitoring scolaire national ÉpStan dans le contexte de la pandémie de COVID-19

in LUCET; SCRIPT (Eds.) Rapport national sur l’éducation au Luxembourg 2021 (2021)

Using Automatic Item Generation in the context of the Épreuves Standardisées (Épstan): A pilot study on effects of altering item characteristics and semantic embeddings

Scientific Conference (2021, November 11)

Assessing mathematical skills in national school monitoring programs such as the Luxembourgish Épreuves Standardisées (ÉpStan) creates a constant demand of developing high-quality items that is both ... [more ▼]

Assessing mathematical skills in national school monitoring programs such as the Luxembourgish Épreuves Standardisées (ÉpStan) creates a constant demand of developing high-quality items that is both expensive and time-consuming. One approach to provide high-quality items in a more efficient way is Automatic Item Generation (AIG, Gierl, 2013). Instead of creating single items, cognitive item models form the base for an algorithmic generation of a large number of new items with supposedly identical item characteristics. The stability of item characteristics is questionable, however, when different semantic embeddings are used to present the mathematical problems (Dewolf, Van Dooren, & Verschaffel, 2017, Hoogland, et al., 2018). Given culture-specific knowledge differences in students, it is not guaranteed that illustrations showing everyday activities do not differentially impact item difficulty (Martin, et al., 2012). Moreover, the prediction of empirical item difficulties based on theoretical rationales has proved to be difficult (Leighton & Gierl, 2011). This paper presents a first attempt to better understand the impact of (a) different semantic embeddings, and (b) problem-related variations on mathematics items in grades 1 (n = 2338), 3 (n = 3835) and 5 (n = 3377) within the context of ÉpStan. In total, 30 mathematical problems were presented in up to 4 different versions, either using different but equally plausible semantic contexts or altering the problem’s content characteristics. Preliminary results of IRT-scaling and DIF-analysis reveal substantial effects of both, the embedding, as well as the problem characteristics on general item difficulties as well as on subgroup level. Further results and implications for developing mathematic items, and specifically, for using AIG in the course of Épstan will be discussed. [less ▲]