Article (Scientific journals)
Contrasting Classical and Machine Learning Approaches in the Estimation of Value-Added Scores in Large-Scale Educational Data
Levy, Jessica; Mussack, Dominic; Brunner, Martin et al.
2020In Frontiers in Psychology, 11, p. 2190
Peer Reviewed verified by ORBi
 

Files


Full Text
Levy et al. - 2020 - Contrasting Classical and Machine Learning Approac.pdf
Publisher postprint (1.77 MB)
Download

All documents in ORBilu are protected by a user license.

Send to



Details



Keywords :
value-added modeling; school effectiveness; machine learning; model comparison; longitudinal data
Abstract :
[en] There is no consensus on which statistical model estimates school value-added (VA) most accurately. To date, the two most common statistical models used for the calculation of VA scores are two classical methods: linear regression and multilevel models. These models have the advantage of being relatively transparent and thus understandable for most researchers and practitioners. However, these statistical models are bound to certain assumptions (e.g., linearity) that might limit their prediction accuracy. Machine learning methods, which have yielded spectacular results in numerous fields, may be a valuable alternative to these classical models. Although big data is not new in general, it is relatively new in the realm of social sciences and education. New types of data require new data analytical approaches. Such techniques have already evolved in fields with a long tradition in crunching big data (e.g., gene technology). The objective of the present paper is to competently apply these “imported” techniques to education data, more precisely VA scores, and assess when and how they can extend or replace the classical psychometrics toolbox. The different models include linear and non-linear methods and extend classical models with the most commonly used machine learning methods (i.e., random forest, neural networks, support vector machines, and boosting). We used representative data of 3,026 students in 153 schools who took part in the standardized achievement tests of the Luxembourg School Monitoring Program in grades 1 and 3. Multilevel models outperformed classical linear and polynomial regressions, as well as different machine learning models. However, it could be observed that across all schools, school VA scores from different model types correlated highly. Yet, the percentage of disagreements as compared to multilevel models was not trivial and real-life implications for individual schools may still be dramatic depending on the model type used. Implications of these results and possible ethical concerns regarding the use of machine learning methods for decision-making in education are discussed.
Disciplines :
Education & instruction
Author, co-author :
Levy, Jessica ;  University of Luxembourg > Faculty of Language and Literature, Humanities, Arts and Education (FLSHASE) > Luxembourg Centre for Educational Testing (LUCET)
Mussack, Dominic ;  University of Luxembourg > Faculty of Language and Literature, Humanities, Arts and Education (FLSHASE) > Education, Culture, Cognition and Society (ECCS)
Brunner, Martin;  University of Potsdam
Keller, Ulrich ;  University of Luxembourg > Faculty of Language and Literature, Humanities, Arts and Education (FLSHASE) > Luxembourg Centre for Educational Testing (LUCET)
Cardoso-Leite, Pedro ;  University of Luxembourg > Faculty of Language and Literature, Humanities, Arts and Education (FLSHASE) > Education, Culture, Cognition and Society (ECCS)
Fischbach, Antoine  ;  University of Luxembourg > Faculty of Language and Literature, Humanities, Arts and Education (FLSHASE) > Luxembourg Centre for Educational Testing (LUCET)
External co-authors :
yes
Language :
English
Title :
Contrasting Classical and Machine Learning Approaches in the Estimation of Value-Added Scores in Large-Scale Educational Data
Publication date :
2020
Journal title :
Frontiers in Psychology
ISSN :
1664-1078
Publisher :
Frontiers Media S.A., Pully, Switzerland
Special issue title :
Transdisciplinary Research on Learning and Teaching: Chances and Challenges
Volume :
11
Pages :
Article 2190
Peer reviewed :
Peer Reviewed verified by ORBi
FnR Project :
FNR10921377 - Capitalising On Linguistic Diversity In Education, 2015 (15/01/2017-14/07/2023) - Peter Gilles
Available on ORBilu :
since 21 September 2020

Statistics


Number of views
220 (29 by Unilu)
Number of downloads
84 (7 by Unilu)

Scopus citations®
 
7
Scopus citations®
without self-citations
5
OpenCitations
 
4
WoS citations
 
6

Bibliography


Similar publications



Contact ORBilu