Students' perceptions of instructional quality; personality; experience sampling; idiosyncratic perceptions; linear mixed effects models
Abstract :
[en] Students’ perceptions of instructional quality (SPIQ) are subjective and time-specific to some extent. Yet, they are mostly aggregated across students and assessed at one time point, neglecting student- and lesson-specific variance. The present study examined the role of students’ personality traits in state SPIQ and their relation to perceived lesson-specific learning achievement (i.e., self-reported comprehension). Thereby, we distinguished between idiosyncratic and consensual (classroom) SPIQ. We assessed the three basic dimensions of instructional quality, teacher support, cognitive activation, and classroom management, as state perceptions of all students within classrooms in mathematics’ instruction ( Nobservations = 2681) across three weeks of 372 German secondary school students’ ( Mage = 15.3 years) daily life. Linear mixed effect models revealed (a) that students’ agreeableness and negative emotionality were positively and negatively, respectively, related to state SPIQ, (b) particularly pronounced positive relations between teacher support and perceived learning achievement, which were (c) stronger for lower levels in agreeableness. Differences across idiosyncratic and consensual perceptions could hardly be detected. Thus, the present study shed light on personality traits’ relations to SPIQ and within-student SPIQ–learning achievement associations, while demonstrating a new application for classroom-based state SPIQ that bridges the gap between intra- and intersubjective perceptions of instructional behavior.
Disciplines :
Education & instruction
Author, co-author :
TALIC, Irma ; University of Luxembourg > Faculty of Humanities, Education and Social Sciences > Department of Behavioural and Cognitive Sciences > Team Christoph NIEPEL ; University of the Bundeswehr Munich, Neubiberg, Germany
Rauthmann, JF; Ludwig-Maximilian-University of Munich, Munich, Germany
Renner, KH; University of the Bundeswehr Munich, Neubiberg, Germany
Aarts E., Verhage M., Veenvliet J. V., Dolan C. V., van der Sluis S. (2014). A solution to dependency: Using multilevel analysis to accommodate nested data. Nature Neuroscience, 17(4), 491–496. https://doi.org/10.1038/nn.3648
Abrahams L., Rauthmann J. F., Fruyt F. D. (2021). Person-situation dynamics in educational contexts: A self- and other-rated experience sampling study of teachers’ states, traits, and situations. European Journal of Personality, 35(4), 598–622. https://doi.org/10.1177/08902070211005621
Amthauer R. (1970). Intelligenz-Struktur-Test [Intelligence structure test]. Hogrefe.
Arens A. K., Marsh H. W., Pekrun R., Lichtenfeld S., Murayama K., vom Hofe R. (2017). Math self-concept, grades, and achievement test scores: Long-term reciprocal effects across five waves and three achievement tracks. Journal of Educational Psychology, 109(5), 621–634. https://doi.org/10.1037/edu0000163
Bates D., Mächler M., Bolker B., Walker S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01
Baumert A., Schmitt M., Perugini M., Johnson W., Blum G., Borkenau P., Costantini G., Denissen J. J. A., Fleeson W., Grafton B., Jayawickreme E., Kurzius E., MacLeod C., Miller L. C., Read S. J., Roberts B., Robinson M. D., Wood D., Wrzus C. (2017). Integrating personality structure, personality process, and personality development. European Journal of Personality, 31(5), 503–528. https://doi.org/10.1002/per.2115
Bellens K., Van Damme J., Van Den Noortgate W., Wendt H., Nilsen T. (2019). Instructional quality: Catalyst or pitfall in educational systems’ aim for high achievement and equity? An answer based on multilevel SEM analyses of TIMSS 2015 data in Flanders (Belgium), Germany, and Norway. Large-Scale Assessments in Education, 7(1), 1–27. https://doi.org/10.1186/s40536-019-0069-2
Ben-Shachar M., Lüdecke D., Makowski D. (2020). effectsize: Estimation of effect size indices and standardized parameters. Journal of Open Source Software, 5(56), 2815. https://doi.org/10.21105/joss.02815
Bernardin H. J., Cooke D. K., Villanova P. (2000). Conscientiousness and agreeableness as predictors of rating leniency. Journal of Applied Psychology, 85(2), 232–236. https://doi.org/10.1037/0021-9010.85.2.232
Bernardin H. J., Tyler C. L., Villanova P. (2009). Rating level and accuracy as a function of rater personality. International Journal of Selection and Assessment, 17(3), 300–310. https://doi.org/10.1111/j.1468-2389.2009.00472.x
Bolger N., Laurenceau J.-P. (2013). Intensive longitudinal methods: An introduction to diary and experience sampling research. Guilford Press.
Breil S. M., Geukes K., Wilson R. E., Nestler S., Vazire S., Back M. D. (2019). Zooming into real-life extraversion – How personality and situation shape sociability in social interactions. Collabra: Psychology, 5(1), 7. https://doi.org/10.1525/collabra.170
Buchmann C., DiPrete T. A., McDaniel A. (2008). Gender inequalities in education. Annual Review of Sociology, 34(1), 319–337. https://doi.org/10.1146/annurev.soc.34.040507.134719
Buckley P., Doyle E. (2017). Individualising gamification: An investigation of the impact of learning styles and personality traits on the efficacy of gamification using a prediction market. Computers & Education, 106, 43–55. https://doi.org/10.1016/j.compedu.2016.11.009
Ching C. M., Church A. T., Katigbak M. S., Reyes J. A. S., Tanaka-Matsumi J., Takaoka S., Zhang H., Shen J., Arias R. M., Rincon B. C., Ortiz F. A. (2014). The manifestation of traits in everyday behavior and affect: A five-culture study. Journal of Research in Personality, 48(1), 1–16. https://doi.org/10.1016/j.jrp.2013.10.002
Costa P. T., McCrae R. R. (1992). Normal personality assessment in clinical practice: The NEO Personality Inventory. Psychological Assessment, 4(1), 5–13. https://doi.org/10.1037//1040-3590.4.1.5
Danner D., Rammstedt B., Bluemke M., Lechner C., Berres S., Knopf T., Soto C. J., John O. P. (2019). Die deutsche Version des Big Five Inventory 2 (BFI-2). Diagnostica, 65(3), 121–132. https://doi.org/10.1026/0012-1924/a000218
Dickhäuser O., Plenter I. (2005). “Letztes Halbjahr stand ich zwei”: Zur Akkuratheit selbst berichteter Noten [On the accuracy of self-reported school marks]. Zeitschrift Für Pädagogische Psychologie [German Journal of Educational Psychology], 19(4), 219–224. https://doi.org/10.1024/1010-0652.19.4.219
Enders C. K., Tofighi D. (2007). Centering predictor variables in cross-sectional multilevel models: A new look at an old issue. Psychological Methods, 12(2), 121–138. https://doi.org/10.1037/1082-989X.12.2.121
Fauth B., Decristan J., Rieser S., Klieme E., Büttner G. (2014). Student ratings of teaching quality in primary school: Dimensions and prediction of student outcomes. Learning and Instruction, 29(1), 1–9. https://doi.org/10.1016/j.learninstruc.2013.07.001
Feistauer D., Richter T. (2017). How reliable are students’ evaluations of teaching quality? A variance components approach. Assessment & Evaluation in Higher Education, 42(8), 1263–1279. https://doi.org/10.1080/02602938.2016.1261083
Fleeson W., Jayawickreme E. (2021). Whole traits: Revealing the social-cognitive mechanisms constituting personality’s central variable. In Gawronski B. (Ed.), Advances in experimental social psychology (pp. 69–128). Elsevier Academic Press. https://doi.org/10.1016/bs.aesp.2020.11.002
Fletcher J. D. (1992). Individualized systems of instruction. Institute for Defense Analyses.
Franzen P., Arens A. K., Greiff S., van der Westhuizen L., Fischbach A., Wollschläger R., Niepel C. (2022). Developing and validating a short-form questionnaire for the assessment of seven facets of conscientiousness in large-scale assessments. Journal of Personality Assessment, 104(6), 759–773. https://doi.org/10.1080/00223891.2021.1998083
Funder D. C., Ozer D. J. (2019). Evaluating effect size in psychological research: Sense and nonsense. Advances in Methods and Practices in Psychological Science, 2(2), 156–168. https://doi.org/10.1177/2515245919847202
Geldhof G. J., Preacher K. J., Zyphur M. J. (2014). Reliability estimation in a multilevel confirmatory factor analysis framework. Psychological Methods, 19(1), 72–91. https://doi.org/10.1037/a0032138
Götz F. M., Gosling S. D., Rentfrow P. J. (2022). Small effects: The indispensable foundation for a cumulative psychological science. Perspectives on Psychological Science, 17(1), 205–215. https://doi.org/10.1177/1745691620984483
Hardy I., Jonen A., Möller K., Stern E. (2006). Effects of instructional support within constructivist learning environments for elementary school students’ understanding of “floating and sinking”. Journal of Educational Psychology, 98(2), 307–326. https://doi.org/10.1037/0022-0663.98.2.307
Hausen J. E., Möller J., Greiff S., Niepel C. (2022). Students’ personality and state academic self-concept: Predicting differences in mean level and within-person variability in everyday school life. Journal of Educational Psychology, 114(6), 1394–1411. https://doi.org/10.1037/edu0000760
Holzberger D., Philipp A., Kunter M. (2013). How teachers’ self-efficacy is related to instructional quality: A longitudinal analysis. Journal of Educational Psychology, 105(3), 774–786. https://doi.org/10.1037/a0032198
Hox J. J., Moerbeek M., van de Schoot R. (2018). Multilevel analysis: Techniques and applications (3rd ed.). Routledge Taylor & Francis.
Jaekel A.-K., Göllner R., Trautwein U. (2021). How students’ perceptions of teaching quality in one subject are impacted by the grades they receive in another subject: Dimensional comparisons in student evaluations of teaching quality. Journal of Educational Psychology, 113(4), 770–783. https://doi.org/10.1037/edu0000488
Jelley R. B. (2021). Using personality feedback for work-related development and performance improvement: A rapid evidence assessment. Canadian Journal of Behavioural Science/Revue Canadienne des Sciences du Comportement, 53(2), 175–186. https://doi.org/10.1037/cbs0000230
John O. P. (2021). History, measurement, and conceptual elaboration of the Big-Five trait taxonomy: The paradigm matures. In John O. P., Robins R. W. (Eds.), Handbook of personality: Theory and research (pp. 35–82). The Guilford Press.
Jonason P. K., Sherman R. A. (2020). Personality and the perception of situations: The Big Five and Dark Triad traits. Personality and Individual Differences, 163, Article 110081. https://doi.org/10.1016/j.paid.2020.110081
Keller H., Karau S. J. (2013). The importance of personality in students’ perceptions of the online learning experience. Computers in Human Behavior, 29(6), 2494–2500. https://doi.org/10.1016/j.chb.2013.06.007
Klieme E., Schümer G., Knoll S. (2001). Mathematikunterricht in der Sekundarstufe I: “Aufgabenkultur” und Unterrichtsgestaltung [Mathematics instruction at secondary level. Task culture and instructional design]. In Bundesministerium für Bildung und Forschung (Ed.), TIMMS - Impulse für Schule und Unterricht (pp. 43–57). Bundesministerium für Bildung und Forschung.
Komarraju M., Karau S. J., Schmeck R. R., Avdic A. (2011). The Big Five personality traits, learning styles, and academic achievement. Personality and Individual Differences, 51(4), 472–477. https://doi.org/10.1016/j.paid.2011.04.019
Kunter M., Baumert J. (2007). Who is the expert? Construct and criteria validity of student and teacher ratings of instruction. Learning Environments Research, 9(3), 231–251. https://doi.org/10.1007/s10984-006-9015-7
Kunter M., Klusmann U., Baumert J., Richter D., Voss T., Hachfeld A. (2013). Professional competence of teachers: Effects on instructional quality and student development. Journal of Educational Psychology, 105(3), 805–820. https://doi.org/10.1037/a0032583
Liepmann D., Beauducel A., Brocke B., Amthauer R. (2007). Intelligenz-Struktur-Test 2000R [Intelligence structure test 2000R]. Hogrefe.
Liepmann D., Beauducel A., Brocke B., Nettelnstroth W. (2012). Intelligenz-Struktur-Test-Screening [Intelligence structure test-screening]. Hogrefe.
Lorah J. (2018). Effect size measures for multilevel models: Definition, interpretation, and TIMSS example. Large-Scale Assessments in Education, 6(1), 8. https://doi.org/10.1186/s40536-018-0061-2
Lüdtke O., Robitzsch A., Trautwein U., Kunter M. (2009). Assessing the impact of learning environments: How to use student ratings of classroom or school characteristics in multilevel modeling. Contemporary Educational Psychology, 34(2), 120–131. https://doi.org/10.1016/j.cedpsych.2008.12.001
Mammadov S. (2022). Big Five personality traits and academic performance: A meta-analysis. Journal of Personality, 90(2), 222–255. https://doi.org/10.1111/jopy.12663
Mang J., Ustjanzew N., Schiepe-Tiska A., Prenzel M., Sälzer C., Müller K., Gonzaléz Rodríguez E. (2018). PISA 2012 Skalenhandbuch: Dokumentation der Erhebungsinstrumente. Waxmann.
Matz S. C., Gladstone J. J., Stillwell D. (2017). In a world of big data, small effects can still matter: A reply to Boyce, Daly, Hounkpatin, and Wood (2017). Psychological Science, 28(4), 547–550. https://doi.org/10.1177/0956797617697445
Matz S. C., Harari G. M. (2021). Personality-place transactions: Mapping the relationships between Big Five personality traits, states, and daily places. Journal of Personality and Social Psychology, 120(5), 1367–1385. https://doi.org/10.1037/pspp0000297
McCrae R. R., Costa P. T. (1987). Validation of the five-factor model of personality across instruments and observers. Journal of Personality and Social Psychology, 52(1), 81–90. https://doi.org/10.1037//0022-3514.52.1.81
Molenaar P. C. M. (2004). A manifesto on psychology as idiographic science: Bringing the person back into scientific psychology, this time forever. Measurement: Interdisciplinary Research & Perspective, 2(4), 201–218. https://doi.org/10.1207/s15366359mea0204_1
Murayama K., Goetz T., Malmberg L.-E., Pekrun R., Tanaka A., Martin A. J. (2017). Within-person analysis in educational psychology: Importance and illustration. In Putwain D. W., Smart K. (Eds.), British Journal of Educational Psychology Monograph Series II: Psychological aspects of education - Current trends: The role of competence beliefs in teaching and learning (pp. 71–87). Wiley.
Muthén L. K., Muthén B. O. (1998-2017). Mplus user’s guide (8th ed.). Muthén & Muthén.
Nakagawa S., Schielzeth H. (2013). A general and simple method for obtaining R2 from generalized linear mixed-effects models. Methods in Ecology and Evolution, 4(2), 133–142. https://doi.org/10.1111/j.2041-210x.2012.00261.x
Niepel C., Marsh H. W., Guo J., Pekrun R., Möller J. (2022). Revealing dynamic relations between mathematics self-concept and perceived achievement from lesson to lesson: An experience-sampling study. Journal of Educational Psychology, 114(6), 1380–1393. https://doi.org/10.1037/edu0000716
Niepel C., Stadler M., Greiff S. (2019). Seeing is believing: Gender diversity in STEM is related to mathematics self-concept. Journal of Educational Psychology, 111(6), 1119–1130. https://doi.org/10.1037/edu0000340
OECD. (2015). The ABC of gender equality in education: Aptitude, behaviour, confidence. PISA. OECD. https://doi.org/10.1787/9789264229945-en
Peterson S. E., Miller J. A. (2004). Quality of college students’ experiences during cooperative learning. Social Psychology of Education, 7(2), 161–183. https://doi.org/10.1023/B:SPOE.0000018522.39515.19
Poropat A. E. (2009). A meta-analysis of the five-factor model of personality and academic performance. Psychological Bulletin, 135(2), 322–338. https://doi.org/10.1037/a0014996
Praetorius A.-K., Klieme E., Herbert B., Pinger P. (2018). Generic dimensions of teaching quality: The German framework of three basic dimensions. ZDM, 50(3), 407–426. https://doi.org/10.1007/s11858-018-0918-4
Randall R., Sharples D. (2012). The impact of rater agreeableness and rating context on the evaluation of poor performance. Journal of Occupational and Organizational Psychology, 85(1), 42–59. https://doi.org/10.1348/2044-8325.002002
Rauthmann J. F. (2021). Capturing interactions, correlations, fits, and transactions: A person-environment relations model. In Rauthmann J. F. (Ed.), The handbook of personality dynamics and processes (pp. 427–522). Elsevier Academic Press. https://doi.org/10.1016/B978-0-12-813995-0.00018-2
Rauthmann J. F., Gallardo-Pujol D., Guillaume E. M., Todd E., Nave C. S., Sherman R. A., Ziegler M., Jones A. B., Funder D. C. (2014). The Situational Eight DIAMONDS: A taxonomy of major dimensions of situation characteristics. Journal of Personality and Social Psychology, 107(4), 677–718. https://doi.org/10.1037/a0037250
Rauthmann J. F., Sherman R. (2019). Toward a research agenda for the study of situation perceptions: A variance componential framework. Personality and Social Psychology Review, 23(3), 238–266. https://doi.org/10.1177/1088868318765600
Rauthmann J. F., Sherman R. A., Nave C. S., Funder D. C. (2015). Personality-driven situation experience, contact, and construal: How people’s personality traits predict characteristics of their situations in daily life. Journal of Research in Personality, 55, 98–111. https://doi.org/10.1016/j.jrp.2015.02.003
R Core Team. (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing.
Roloff J., Klusmann U., Lüdtke O., Trautwein U. (2020). The predictive validity of teachers’ personality, cognitive and academic abilities at the end of high school on instructional quality in Germany: A longitudinal study. AERA Open, 6(1), 1–17. https://doi.org/10.1177/2332858419897884
Ruzek E., Aldrup K., Lüdtke O. (2022). Assessing the effects of student perceptions of instructional quality: A cross-subject within-student design. Contemporary Educational Psychology, 70, Article 102085. https://doi.org/10.1016/j.cedpsych.2022.102085
Ryan R. M., Deci E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55(1), 68–78. https://doi.org/10.1037/0003-066X.55.1.68
Scherer R., Nilsen T., Jansen M. (2016). Evaluating individual students’ perceptions of instructional quality: An investigation of their factor structure, measurement invariance, and relations to educational outcomes. Frontiers in Psychology, 7, 110. https://doi.org/10.3389/fpsyg.2016.00110
Seidel T., Shavelson R. J. (2007). Teaching effectiveness research in the past decade: The role of theory and research design in disentangling meta-analysis results. Review of Educational Research, 77(4), 454–499. https://doi.org/10.3102/0034654307310317
Sherman R. A., Rauthmann J. F., Brown N. A., Serfass D. G., Jones A. B. (2015). The independent effects of personality and situations on real-time expressions of behavior and emotion. Journal of Personality and Social Psychology, 109(5), 872–888. https://doi.org/10.1037/pspp0000036
Shernof D. J., Ruzek E. A., Sannella A. J., Schorr R. Y., Sanchez-Wall L., Bressler D. M. (2017). Student engagement as a general factor of classroom experience: Associations with student practices and educational outcomes in a university gateway course. Frontiers in Psychology, 8, 994. https://doi.org/10.3389/fpsyg.2017.00994
Soto C. J., John O. P. (2017). The next Big Five Inventory (BFI-2): Developing and assessing a hierarchical model with 15 facets to enhance bandwidth, fidelity, and predictive power. Journal of Personality and Social Psychology, 113(1), 117–143. https://doi.org/10.1037/pspp0000096
Sparfeldt J. R., Buch S. R., Rost D. H., Lehmann G. (2008). Akkuratesse selbstberichteter Zensuren [The accuracy of self-reported grades in school]. Psychologie in Erziehung und Unterricht, 55(1), 68–75.
Spooren P., Brockx B., Mortelmans D. (2013). On the validity of student evaluation of teaching: The state of the art. Review of Educational Research, 83(4), 598–642. https://doi.org/10.3102/0034654313496870
Sutin A. R., Stephan Y., Luchetti M., Strickhouser J. E., Aschwanden D., Terracciano A. (2022). The association between five factor model personality traits and verbal and numeric reasoning. Aging, Neuropsychology, and Cognition, 29(2), 297–317. https://doi.org/10.1080/13825585.2021.1872481
Talić I., Scherer R., Marsh H. W., Greiff S., Möller J., Niepel C. (2022). Uncovering everyday dynamics in students’ perceptions of instructional quality with experience sampling. Learning and Instruction, 81, Article 101594. https://doi.org/10.1016/j.learninstruc.2022.101594
Toropova A., Johansson S., Myrberg E. (2019). The role of teacher characteristics for student achievement in mathematics and student perceptions of instructional quality. Education Inquiry, 10(4), 275–299. https://doi.org/10.1080/20004508.2019.1591844
Tsai Y.-M., Kunter M., Lüdtke O., Trautwein U., Ryan R. M. (2008). What makes lessons interesting? The role of situational and individual factors in three school subjects. Journal of Educational Psychology, 100(2), 460–472. https://doi.org/10.1037/0022-0663.100.2.460
UNESCO. (2017). Cracking the code: Girls’ and women’s education in science, tech-nology, engineering and mathematics (STEM). UNESCO.
Vieluf S., Praetorius A.-K., Rakoczy K., Kleinknecht M., Pietsch M. (2020). Angebots-Nutzungs-Modelle der Wirkweise des Unterrichts: Ein kritischer Vergleich verschiedener Modellvarianten. Zeitschrift für Padagogik, 66(1), 63–80. https://doi.org/10.3262/ZPB2001063
Wagner W., Göllner R., Helmke A., Trautwein U., Lüdtke O. (2013). Construct validity of student perceptions of instructional quality is high, but not perfect: Dimensionality and generalizability of domain-independent assessments. Learning and Instruction, 28, 1–11. https://doi.org/10.1016/j.learninstruc.2013.03.003
Wagner W., Göllner R., Werth S., Voss T., Schmitz B., Trautwein U. (2016). Student and teacher ratings of instructional quality: Consistency of ratings over time, agreement, and predictive power. Journal of Educational Psychology, 108(5), 705–721. https://doi.org/10.1037/edu0000075
Weisberg Y. J., DeYoung C. G., Hirsh J. B. (2011). Gender differences in personality across the ten aspects of the big five. Frontiers in Psychology, 2, 178. https://doi.org/10.3389/fpsyg.2011.00178
Wilms R., Mäthner E., Winnen L., Lanwehr R. (2021). Omitted variable bias: A threat to estimating causal relationships. Methods in Psychology, 5, Article 100075. https://doi.org/10.1016/j.metip.2021.100075
Wisniewski B., Röhl S., Fauth B. (2022). The perception problem: A comparison of teachers’ self-perceptions and students’ perceptions of instructional quality. Learning Environments Research, 25(3), 775–802. https://doi.org/10.1007/s10984-021-09397-4
Wisniewski B., Zierer K., Dresel M., Daumiller M. (2020). Obtaining secondary students’ perceptions of instructional quality: Two-level structure and measurement invariance. Learning and Instruction, 66, Article 101303. https://doi.org/10.1016/j.learninstruc.2020.101303
Yun G. J., Donahue L. M., Dudley N. M., McFarland L. A. (2005). Rater personality, rating format, and social context: Implications for performance appraisal ratings. International Journal of Selection and Assessment, 13(2), 97–107. https://doi.org/10.1111/j.0965-075X.2005.00304.x
Zhang L.-F. (2003). Does the big five predict learning approaches? Personality and Individual Differences, 34(8), 1431–1446. https://doi.org/10.1016/S0191-8869(02)00125-3
