The role of engagement and academic behavioral skills on young students’ academic performance—A validation across four countries

Siddiq, F; Gochyyev, P; VALLS CASAS, Ona

doi:10.1016/j.stueduc.2020.100880

Article (Scientific journals)

The role of engagement and academic behavioral skills on young students’ academic performance—A validation across four countries

Siddiq, F; Gochyyev, P; VALLS CASAS, Ona

2020 • In Studies in Educational Evaluation, 66

Peer Reviewed verified by ORBi

Permalink
https://hdl.handle.net/10993/58316

DOI
10.1016/j.stueduc.2020.100880

Files (1)Send to Details Statistics Bibliography Similar publications

Files

Full Text

1-s2.0-S0191491X20301280-main.pdf

Author postprint (626.37 kB)

Download

All documents in ORBilu are protected by a user license.

Send to

RIS BibTex APA Chicago Permalink X Linkedin

Details

Keywords :

Academic behavioral skills; Engagement; Item response theory; Rasch modelling; Secondary education; Validation

Abstract :

[en] The aim of this study is to validate an instrument measuring students’ academic behavioral skills and engagement—skills identified as vital for student achievement. We inspect the reliability and validity of the survey with respect to item fit, factorial structure, relations with academic performance, and the fairness of the items across student groups. The fairness analyses are critical to making valid comparisons between groups and across countries. Data comprising 8520 grade 10 students from four countries were analysed using item response theory. We found that both scales were multidimensional, acted fairly across students’ gender, country, immigrant-, and socio-economic background (after removing four items), and were positively and significantly correlated with self-reported and performance-based academic performance. © 2020 Elsevier Ltd

Disciplines :

Sociology & social sciences

Author, co-author :

Siddiq, F

Gochyyev, P

VALLS CASAS, Ona ; University of Luxembourg > Faculty of Humanities, Education and Social Sciences (FHSE) > Department of Social Sciences (DSOC) > Centre for Childhood and Youth Research

External co-authors :

yes

Language :

English

Title :

The role of engagement and academic behavioral skills on young students’ academic performance—A validation across four countries

Publication date :

2020

Journal title :

Studies in Educational Evaluation

ISSN :

0191-491X

Volume :

Peer reviewed :

Peer Reviewed verified by ORBi

Available on ORBilu :

since 30 November 2023

Statistics

Number of views

1 (0 by Unilu)

Number of downloads

109 (0 by Unilu)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenCitations

Bibliography

Adams, R.J., Khoo, S.-T., Quest. 1996, ACER Press, Melbourne, Australia [computer program].
Adams, R.J., Wilson, M., Wang, W., The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement 21:1 (1997), 1–23.
AERA, APA, NCME, Standards for educational and psychological testing. 2014, AERA, Washington, DC.
Aesaert, K., Voogt, J., Kuiper, E., van Braak, J., Accuracy and bias of ICT self-efficacy: An empirical study into students’ over- and underestimation of their ICT competences. Computers in Human Behavior 75 (2017), 92–102, 10.1016/j.chb.2017.05.010.
Appleton, J., Christenson, S., Furlong, M., Student engagement with school: Critical conceptual and methodological issues of the construct. Psychology in the Schools 45 (2008), 369–386, 10.1002/pits.20303.
Baker, F.B., Kim, S.H., Item response theory. Parameter estimation techniques. 2nd ed., 2004, Marcel Dekker.
Bandura, A., Self-efficacy: The exercise of control. 1997, Freeman.
Boekaerts, M., Engagement as an inherent aspect of the learning process. Learning and Instruction 43 (2016), 76–83, 10.1016/j.learninstruc.2016.02.001.
Bol, L., Hacker, D.J., O'Shea, P., Allen, D., The influence of overt practice, achievement level, and exploratory style on calibration accuracy and performance. Journal of Experimental Education 73:4 (2005), 269–290.
Bollen, K.A., Structural equations with latent variables. 1989, Wiley, New York.
Bonal, X., Apropiaciones escolares: entre las posiciones sociales y las disposiciones educativas. 2003, Fundació Jaume Bofill.
Briggs, D.C., Wilson, M., An introduction to multidimensional measurement using Rasch models. Journal of Applied Measurement 4:1 (2003), 87–100.
Caprara, G.V., Fida, R., Vecchione, M., Del Bove, G., Vecchio, G.M., Barbaranelli, C., et al. Longitudinal Analysis of the role of perceived self-efficacy for self-regulated learning in academic continuance and achievement. Journal of Educational Psychology 100 (2008), 525–534.
Chen, P., Exploring the accuracy and predictability of the self-efficacy beliefs of seventh-grade mathematics students. Learning and Individual Differences 14 (2003), 79–92.
Clark, K.N., Malecki, C.K., Academic Grit Scale: Psychometric properties and associations with achievement and life satisfaction. Journal of School Psychology 72 (2019), 49–66, 10.1016/j.jsp.2018.12.001.
Credé, M., Tynan, M., Harms, P.D., Much ado about grit: A meta-analytic synthesis of the grit literature. Journal of Personality and Social Psychology 113:3 (2017), 492–511, 10.1037/pspp0000102.
de Ayala, R.J., The IRT tradition and its applications. Little, T.D., (eds.) The Oxford handbook of quantitative methods, Vol. 1, 2013, Oxford University Press, 144–169.
DeVellis, R.F., Scale development: Theory and applications. 2012, Sage Publications.
Dorans, N.J., Holland, P.W., DIF detection and description: Mantel-Haenszel and standardization. Holland, P.W., Wainer, H., (eds.) Differential item functioning, 1993, Lawrence Erlbaum Associates, 35–66.
Duckworth, A.L., Tsukayama, E., May, H., Establishing causality using longitudinal hierarchical linear modeling: An illustration predicting achievement from self-control. Social Psychological and Personality Science 1:4 (2010), 311–317.
Duckworth, A.L., Peterson, C., Matthews, M.D., Kelly, D.R., Grit: Perseverance and passion for long term goals. Journal of Personality and Social Psychology 92:6 (2007), 1087–1101, 10.1037/0022-3514.92.6.1087.
Dweck, C.S., Mindset: The new psychology of success. 2006, Random House Publishing.
Farrington, C.A., Academic mindsets as a critical component of deeper learning. 2013, University of Chicago: Consortium on Chicago School Research. A White Paper prepared for the William and Flora Hewlett Foundation https://www.hewlett.org/wp-content/uploads/2016/08/Academic_Mindsets_as_a_Critical_Component_of_Deeper_Learning_CAMILLE_FARRINGTON_April_20_2013.pdf.
Farrington, C.A., Roderick, M., Allensworth, E., Nagaoka, J., Keyes, T.S., Johnson, D.W., et al. Teaching adolescents to become learners. The role of noncognitive factors in shaping school performance: A critical literature review. 2012, University of Chicago Consortium on Chicago School Research, Chicago.
Finn, J., Rock, D., Academic success among students at risk for school failure. The Journal of Applied Psychology 82:2 (1997), 221–234, 10.1037/0021-9010.82.2.221.
Finn, J., Voelkl, K., School characteristics related to student engagement. The Journal of Negro Education, 1993, 10.2307/2295464.
Finn, J., Zimmer, K., Student engagement: What is it? Why does it matter?. Christenson, S.L., Reschly, A.L., Wylie, C., (eds.) Handbook of research on student engagement, 2012, Springer Science + Business Media, 97–131, 10.1007/978-1-4614-2018-7_5.
Fredricks, J.A., McColskey, W., The measurement of student engagement: A comparative analysis of various methods and student self-report instruments. Christenson, S.L., Reschly, A.L., Wylie, C., (eds.) Handbook of research on student engagement, 2012, Springer Science + Business Media, 763–782, 10.1007/978-1-4614-2018-7_37.
Fredricks, J., Blumenfeld, P., Paris, A., School engagement: Potential of the concept, state of the evidence. Review of Educational Research 74:1 (2004), 59–109, 10.3102/00346543074001059.
Glanville, L., Wildhagen, T., The measurement of school engagement. Assessing dimensionality and measurement invariance across race and ethnicity. Educational and Psychological Measurement 67:6 (2007), 1019–1041, 10.1177/0013164406299126.
Green, J., Liem, G., Martin, A., Colmar, S., Marsh, H., McInerney, D., Academic motivation, self-concept, engagement, and performance in high school: Key processes from a longitudinal perspective. Journal of Adolescence 35 (2012), 1111–1122, 10.1016/j.adolescence.2012.02.016.
Gresham, F.M., Elliott, S.N., The social skills rating system. 1990, American Guidance Service.
Griffin, P., Care, E., The ATC21S method. Griffin, P., Care, E., (eds.) Assessment and teaching of 21st century skills: Methods and approaches, 2015, Springer, 3–33.
Griffin, P., McGaw, B., Care, E., (eds.) Assessment and teaching of 21st century skills, 2012, Springer.
Gutman, L.M., Schoon, I., The impact of non-cognitive skills on outcomes for young people, Literature review. 2013, The Education Endowment Foundation (EEF). Institute of education https://educationendowmentfoundation.org.uk/public/files/Publications/EEF_Lit_Review_Non-CognitiveSkills.pdf.
Hambleton, R.K., Jones, R.W., Comparison of classical test theory and item response theory and their applications to test development. Educational Measurement Issues and Practice 12 (1993), 535–556.
Hambleton, R.K., Swaminathan, H., Rogers, H.J., Fundamentals of item response theory. 1991, Sage.
Hu, L., Bentler, P.M., Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling A Multidisciplinary Journal 6:1 (1999), 1–55, 10.1080/10705519909540118.
Janosz, M., Archambault, I., Morizot, J., Pagani, L., School engagement trajectories and their differential predictive relations to dropout. The Journal of Social Issues 64:1 (2008), 21–40, 10.1111/j.1540-4560.2008.00546.x.
Jimerson, S., Campos, E., Green, J., Toward an understanding of definitions and measures of school engagement and related terms. The California School Psychologist: CASP / California Association of School Psychologists 8 (2003), 7–27, 10.1007/BF03340893.
John, O.P., Srivastava, S., The Big Five trait taxonomy: History, measurement, and theoretical perspectives. Pervin, L.A., John, O.P., (eds.) Handbook of personality: Theory and research, 1999, Guilford Press, 102–138.
Khine, M.S., Areepattamannil, S., Non-cognitive skills and factors in educational attainment. Contemporary approaches to research in learning innovations, 2016, Springer.
Kline, R.B., Principles and practice of structural equation modeling. 2005, Guilford Press, New York.
Lamb, S., Jackson, J., Rumberger, R., ISCY technical paper: Measuring 21st century skills in ISCY. Technical Report, 2015, Victoria University, Centre for International Research on Educational Systems, Melbourne, Victoria.
Lee, J.S., The relationship between student engagement and academic performance: Is it a myth or reality?. The Journal of Educational Research 107:3 (2014), 177–185, 10.1080/00220671.2013.807491.
Libbey, H., Measuring student relationships to school: Attachment, bonding, connectedness, and engagement. The Journal of School Health 74:7 (2004), 274–283, 10.1111/j.1746-1561.2004.tb08284.x.
Martin, J.H., Montgomery, R.L., Saphian, D., Personality, achievement test scores, and high school percentile as predictors of academic performance across four years of coursework. Journal of Research in Personality 40 (2006), 424–431.
Masters, G.N., A Rasch model for partial credit scoring. Psychometrika 47:2 (1982), 149–174.
Messick, S., Validity of psychological assessment. Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. The American Psychologist 50 (1995), 741–749, 10.1037/0003-066X.50.9.741.
Miller, T.M., Geraci, L., Unskilled but aware: Reinterpreting overconfidence in low-performing students. Journal of Experimental Psychology Learning, Memory, and Cognition 37:2 (2011), 502–506, 10.1037/a0021802.
Millsap, R.E., Statistical approaches to measurement invariance. 2011, Routledge.
Moffitt, T.E., Arseneault, L., Belsky, D., Dickson, N., Hancox, R.J., Harrington, H., et al. A gradient of childhood self-control predicts health, wealth, and public safety. Proceedings of the National Academy of Sciences 108:7 (2011), 2693–2698.
Muthén, B., Muthén, L., Mplus 7.3. [Computer Software], 2014, Muthén & Muthén.
OECD, Skills for social progress: The power of social and emotional skills. OECD skills studies. 2015, OECD Publishing https://www.oecd-ilibrary.org/education/skills-for-social-progress_9789264226159-en.
OECD, PISA 2006. Technical report, 2006, OECD http://www.oecd.org/dataoecd/0/47/42025182.pdf.
Palardy, G.J., Rumberger, R.W., The effects of 21st century skills on behavioral disengagement in Sacramento high schools. Demanet, J., Van Houtte, M., (eds.) Resisting education: A cross-national study on systems and school effects, 2019, Springer, 53–80.
Palardy, G.J., Rumberger, R.W., Butler, T., The effect of high school socioeconomic, racial, and linguistic segregation on academic performance and school behaviors. Teachers College Record 117:12 (2015), 1–52.
Partnership for 21st Century Skills (P21) and AACTE (American Association of Colleges of Teacher Education), 21st century knowledge and skills in educator preparation. White paper, 2010 http://www.p21.org/storage/documents/aacte_p21_whitepaper2010.pdf.
Pellegrino, J.W., Hilton, M.L., (eds.) Education for life and work: Developing transferable knowledge and skills in the 21st century, 2012, National Research Council. The National Academies Press.
Rabe-Hesketh, S., Skrondal, A., Multilevel and longitudinal modeling using STATA. 2005, Stata Press.
Rasch, G., Probabilistic models for some intelligence and achievement tests. 1960, Danish Institute for Educational Research. MESA Press (Expanded edition 1983. ed.).
Reeve, J., Tseng, C., Agency as a fourth aspect of students’ engagement during learning activities. Contemporary Educational Psychology 36:4 (2011), 257–267, 10.1016/j.cedpsych.2011.05.002.
Renninger, K.A., Hidi, S.E., The power of interest for motivation and engagement. 2016, Routledge/Taylor & Francis Group.
Robbins, S.B., Lauver, K., Le, H., Davis, D., Langley, R., Carlstrom, A., Do psychosocial and study skill factors predict college outcomes? A meta-analysis. Psychological Bulletin 130 (2004), 261–288.
Ross, S., Motivation correlates of academic achievement: Exploring how motivation influences academic achievement in the PISA 2003 dataset. [Doctoral dissertation, University of Victoria], 2008 http://dspace.library.uvic.ca/bitstream/handle/1828/3209/Ross_Shelley_PhD_2008.pdf?sequence=1&isAllowed=y.
Rotermund, R., The role of psychological antecedents and student engagement in a process model of high school dropout. [Doctoral dissertation, University of California Santa Barbara], 2010 https://eric.ed.gov/?id=ED522153.
Siddiq, F., Gochyyev, P., Wilson, M., Learning in digital networks – ICT literacy: A novel assessment of students’ 21st century skills. Computers & Education 109 (2017), 11–37, 10.1016/j.compedu.2017.01.014.
Skinner, E., Kindermann, T., Furrer, C., A motivational perspective on engagement and disaffection: Conceptualization and assessment of children's behavioral and emotional participation in academic activities in the classroom. Educational and Psychological Measurement 69:3 (2009), 493–525, 10.1177/0013164408323233.
Südkamp, A., Kaiser, J., Möller, J., Accuracy of teachers’ judgments of students’ academic achievement: A meta-analysis. Journal of Educational Psychology 104:3 (2012), 743–762, 10.1037/a0027627.
Te Wang, M., Fredricks, J., Ye, F., Hofkens, T., Linn, J.S., Conceptualization and assessment of adolescents’ engagement and disengagement in school: A multidimensional school engagement scale. European Journal of Psychological Assessment 35:4 (2019), 592–606, 10.1027/1015-5759/a000431.
Thissen, D., Marginal maximum likelihood estimation for the one-parameter logistic model. Psychometrika 47 (1982), 175–186.
Thomas, M.L., The value of item response theory in clinical assessment: A review. Assessment 18 (2011), 291–307.
Voogt, J., Roblin, N.P., A comparative analysis of international frameworks for 21st century competences: Implications for national curriculum policies. Journal of Curriculum Studies 44 (2012), 299–332.
Wang, M., Willett, J., Eccles, J., The assessment of school engagement: Examining dimensionality and measurement invariance by gender and race/ethnicity. Journal of School Psychology 49:4 (2011), 465–480, 10.1016/j.jsp.2011.04.001.
Wilson, M., Constructing Measures: An item response modeling approach. 2005, Lawrence Erlbaum Associates, Inc.
Wilson, M., Gochyyev, P., Psychometrics. Teo, T., (eds.) Handbook of quantitative methods for educational research, 2013, Springer, 1–53.
Wu, M.L., Adams, R.J., Properties of Rasch residual fit statistics. Journal of Applied Measurement 14:4 (2013), 339–355.
Wu, M., Adams, R., Wilson, M., Haldane, S., ConQuest: Generalised item response modelling software (Version 2.0). 2007, ACER Press.