[en] In psychological assessment, gauging the impact of personality traits on academic outcomes is vital. Many studies explore the relation between academic achievement and traits like conscientiousness but prioritize description over prediction. Addressing this gap by focusing on actual prediction can refine assessment methodologies and deepen theoretical understanding. Our study focuses on predicting the influence of conscientiousness facets on standardized test scores using various machine learning strategies. Data from N = 7,949 Luxembourgish Grade 9 students showed a gradient boosting model with item-level predictors outperformed traditional linear regression (R2 = .123 vs. R2 = .077). This model revealed both linear and nonlinear ties between conscientiousness facets and achievement. Our findings accentuate conscientiousness’s underestimated predictive power for academic success and advocate for machine learning as a pivotal tool in psychological testing, particularly for outcome prediction.
Centre de recherche :
Faculty of Language and Literature, Humanities, Arts and Education (FLSHASE) > Luxembourg Centre for Educational Testing (LUCET) Ludwig-Maximilian University of Munich, Germany DIPF | Leibniz Institute for Research and Information in Education, Germany Institute of Medical Education, LMU University Hospital, LMU Munich, Germany Technical University Munich & Centre for International Student Assessment, Germany
Bißantz, S., Frick, S., Melinscak, F., Iliescu, D., & Wetzel, E. (2024). The potential of machine learning methods in psychological assessment and test construction. European Journal of Psychological Assessment, 40 (1), 1-4. https://doi.org/10.1027/1015-5759/a000817
Borghans, L., Golsteyn, B. H. H., Heckman, J. J., & Humphries, J. E. (2016). What grades and achievement tests measure. Proceedings of the National Academy of Sciences of the United States of America, 113 (47), 13354-13359. https://doi.org/10.1073/pnas.1601135113
Breiman, L. (2001). Random Forests. Machine Learning, 45, 5-32. https://doi.org/10.1023/A:1010933404324
Bleidorn, W., Hill, P. L., Back, M. D., Denissen, J. J. A., Hennecke, M., Hopwood, C. J., Jokela, M., Kandler, C., Lucas, R. E., Luhmann, M., Orth, U., Wagner, J., Wrzus, C., Zimmermann, J., & Roberts, B. (2019). The policy relevance of personality traits. American Psychologist, 74 (9), 1056-1067. https://doi.org/10.1037/amp0000503
Chen, T., & Guestrin, C. (2016). XGBoost. In B. Krishnapuram, M. Shah, A. Smola, C. Aggarwal, D. Shen, & R. Rastogi (Eds.), Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 785-794). ACM. https://doi.org/10.1145/2939672.2939785
Chen, T., He, T., Benesty, M., Khotilovich, V., Tang, Y., Cho, H., Chen, K., Mitchell, K., Cano, I., Zhou, T., Li, M., Xie, J., Lin, M., Geng, Y., & Li, Y. (2021). xgboost: Extreme gradient boosting (R package version 1.5.0.1) https://CRAN.R-project.org/package=xgboost
Chernikova, O., Franzen, P., Arens, A. K., Niepel, C., Stadler, M., Keller, U., Fischbach, A., & Greiff, S. (2024, July 23). Improving prediction through machine learning: The relation between conscientiousness and academic achievement [Data]. https://osf.io/gjby6
Chicco, D., Warrens, M. J., & Jurman, G. (2021). The coefficient of determination R-squared is more informative than SMAPE, MAE, MAPE, MSE and RMSE in regression analysis evaluation. PeerJournal Computer Science, 7, Article e623. https://doi.org/10.7717/peerj-cs.623
Digman, J. M. (1990). Personality structure: Emergence of the five-factor model. Annual Review of Psychology, 41 (1), 417-440. https://doi.org/10.1146/annurev.ps.41.020190.002221
Drucker, H., Burges, C., Kaufman, L., Smola, A., & Vapnik, V. (1997). Support vector regression machines. Advances in Neural Information Processing Systems, 28 (7), 155-161. https://doi.org/10.5555/2998981.2999003
Fischbach, A., Ugen, S., & Martin, R. (2014). ÉpStan Technical Report. University of Luxembourg. http://orbilu.uni.lu/handle/10993/15802
Franzen, P., Arens, A. K., Greiff, S., van der Westhuizen, L., Fischbach, A., Wollschläger, R., & Niepel, C. (2021). Developing and validating a short-form questionnaire for the assessment of seven facets of conscientiousness in large-scale assessments. Journal of Personality Assessment, 104 (6), 759-773. https://doi.org/10.1080/00223891.2021.1998083
Frey, M. C., & Detterman, D. K. (2004). Scholastic assessment or g? The relationship between the scholastic assessment test and general cognitive ability. Psychological Science, 15 (6), 373-378. https://doi.org/10.1111/j.0956-7976.2004.00687.x
Friedman, J. H. (2001). Greedy function approximation: A gradient boosting machine. Annals of Statistics, 29 (5), 1189-1232. https://doi.org/10.1214/aos/1013203451
Hilbert, S., Coors, S., Kraus, E., Bischl, B., Lindl, A., Frei, M., Wild, J., Krauss, S., Goretzko, D., & Stachl, C. (2021). Machine learning for the educational sciences. Review of Education, 9, Article e3310. https://doi.org/10.1002/rev3.3310
Inoue, A., & Kilian, L. (2005). In-sample or out-of-sample tests of predictability: Which one should we use? Econometric Reviews, 23 (4), 371-402. https://doi.org/10.1081/ETC-200040785
James, G., Witten, D., Hastie, T., & Tibshirani, R. (2021). An introduction to statistical learning with applications in R (2nd ed.). Springer. https://hastie.su.domains/ISLR2/ISLRv2_corrected_June_2023.pdf.download.html
Kuhn, M. (2014). Futility analysis in the cross-validation of machine learning models. https://arxiv.org/abs/1405.6974
Kuhn, M., & Johnson, K. (2013). Applied predictive modeling. Springer.
Kuhn, M., & Wickham, H. (2020). Tidymodels: A collection of packages for modeling and machine learning using tidyverse principles. https://tidymodels.org
Liu, Y., & Just, A. (2020). SHAPforxgboost: SHAP plots for "XGBoost". Version 0.1.1. https://CRAN.R-project.org/package=SHAPforxgboost
LUCET. (n.d.). Specific information: Assessed competencies. https://epstan.lu/en/assessed-competences-9/
Lundberg, S. M., & Lee, S. I. (2017). A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 30, 4765-4774. https://doi.org/10.5555/3295222.3295230
MacCann, C., Duckworth, A. L., & Roberts, R. D. (2009). Empirical identification of the major facets of conscientiousness. Learning and Individual Differences, 19 (4), 451-458. https://doi.org/10.1016/j.jrp.2016.01.002
Martin, R., Ugen, S., & Fischbach, A. (Eds.). (2014). Épreuves Standardisées. Bildungsmonitoring für Luxemburg. Nationaler Bericht 2011-2013 [Épreuves Standardisées: School monitoring for Luxembourg. National report 2011-2013]. https://men.public.lu/de/publications/statistiques-etudes/statistiques-globales/epreuves-standardisees.html
McDonald, R. P. (1999). Test theory: A unified treatment. Erlbaum.
Meyer, J., Jansen, T., Hübner, N., & Lüdtke, O. (2023). Disentangling the association between the Big Five personality traits and student achievement: Meta-Analytic evidence on the role of domain specificity and achievement measures. Educational Psychology Review, 35 (1), 1-34. https://doi.org/10.1007/s10648-023-09736-2
Mõttus, R., Wood, D., Condon, D. M., Back, M. D., Baumert, A., Costantini, G., Epskamp, S., Greiff, S., Johnson, W., Lukaszewski, A., Murray, A., Revelle, W., Wright, A. G. C., Yarkoni, T., Ziegler, M., & Zimmermann, J. (2020). Descriptive, predictive and explanatory personality research: Different goals, different approaches, but a shared need to move beyond the Big Few traits. European Journal of Personality, 34 (6), 1175-1201. https://doi.org/10.1002/per.2311
Noftle, E. E., & Robins, R. W. (2007). Personality predictors of academic outcomes: Big five correlates of GPA and SAT scores. Journal of Personality and Social Psychology, 93 (1), 116-130. https://doi.org/10.1037/0022-3514.93.1.116
Nowok, B., Raab, G. M., & Dibben, C. (2016). synthpop: Bespoke creation of synthetic data in R. Journal of Statistical Software, 74 (11), 1-26. https://doi.org/10.18637/jss.v074.i11
Poropat, A. E. (2009). A meta-Analysis of the five-factor model of personality and academic performance. Psychological Bulletin, 135 (2), 322-338. https://doi.org/10.1037/a0014996
Poropat, A. E. (2014). A meta-Analysis of adult-rated child personality and academic performance in primary education. British Journal of Educational Psychology, 84 (Pt 2), 239-252. https://doi.org/10.1111/bjep.12019
R Core Team. (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org
Rikoon, S. H., Brenneman, M., Kim, L. E., Khorramdel, L., MacCann, C., Burrus, J., & Roberts, R. D. (2016). Facets of conscientiousness and their differential relationships with cognitive ability factors. Journal of Research in Personality, 61, 22-34. https://doi.org/10.1016/j.jrp.2016.01.002
Roberts, B. W., Chernyshenko, O. S., Stark, S. E., & Goldberg, L. R. (2005). The structure of conscientiousness: An empirical investigation based on seven major personality questionnaires. Personnel Psychology, 58 (1), 103-139. https://doi.org/10.1111/j.1744-6570.2005.00301.x
Roberts, B. W., Jackson, J. J., Fayard, J. V., Edmonds, G., & Meints, J. (2009). Conscientiousness. In M. R. Leary, & R. H. Hoyle (Eds.), Handbook of individual differences in social behavior (pp. 369-381). Guilford Press.
Santner, T., Williams, B., & Notz, W. (2003). The design and analysis of computer experiments. Springer.
Seeboth, A., & Mõttus, R. (2018). Successful explanations start with accurate descriptions: Questionnaire items as personality markers for more accurate predictions. European Journal of Personality, 32 (3), 186-201. https://doi.org/10.1002/per.2147
Shewry, M., & Wynn, H. (1987). Maximum Entropy Sampling. Journal of Applied Statistics, 14 (2), 165-170. https://doi.org/10.1080/02664768700000020
Stachl, C., Pargent, F., Hilbert, S., Harari, G. M., Schoedel, R., Vaid, S., Gosling, S. D., & Bühner, M. (2020). Personality research and assessment in the era of machine learning. European Journal of Personality, 34 (5), 613-631. https://doi.org/10.1002/per.2257
Stewart, R. D., Mõttus, R., Seeboth, A., Soto, C. J., & Johnson, W. (2022). The finer details? The predictability of life outcomes from Big Five domains, facets, and nuances. Journal of Personality, 90 (2), 167-182. https://doi.org/10.1111/jopy.12660
Warm, T. A. (1985). Weighted likelihood estimation of ability in Item Response Theory with tests of finite length. The University of Oklahoma. https://shareok.org/handle/11244/5356
Yarkoni, T., & Westfall, J. (2017). Choosing prediction over explanation in psychology: Lessons from machine learning. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 12 (6), 1100-1122. https://doi.org/10.1177/1745691617693393
Zou, H., & Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67 (2), 301-320. https://doi.org/10.1111/j.1467-9868.2005.00503.x
