Developing and Validating a Short-Form Questionnaire for the Assessment of Seven Facets of Conscientiousness in Large-Scale Assessments

in Journal of Personality Assessment (2022), 104(6), 759-773

Conscientiousness is the most important personality predictor of academic achievement. It consists of several lower order facets with differential relations to academic achievement. There is currently no ... [more ▼]

Conscientiousness is the most important personality predictor of academic achievement. It consists of several lower order facets with differential relations to academic achievement. There is currently no short instrument assessing facets of conscientiousness in the educational context. Therefore, in the present multi-study report, we develop and validate a short-form questionnaire for the assessment of seven Conscientiousness facets, namely Industriousness, Perfectionism, Tidiness, Procrastination Refrainment, Control, Caution, and Task Planning. To this end, we examined multiple representative samples totaling N = 14,604 Grade 9 and 10 students from Luxembourg. The questionnaire was developed by adapting and shortening an existing scale using an exhaustive search algorithm. The algorithm was specified to select the best item combination based on model fit, reliability, and measurement invariance across the German and French language versions. The resulting instrument showed the expected factorial structure. The relations of the facets with personality constructs and academic achievement were in line with theoretical assumptions. Reliability was acceptable for all facets. Measurement invariance across language versions, gender, immigration status and cohort was established. We conclude that the presented questionnaire provides a short measurement of seven facets of Conscientiousness with valid and reliable scores. [less ▲]

The development and validation of a short conscientiousness questionnaire for large-scale educational assessment

Scientific Conference (2020, July)

Conscientiousness and its subfacets are related to multiple learning-related outcomes. MacCann, Duckworth and Roberts (2009) developed a questionnaire measuring seven subfacets of conscientiousness with ... [more ▼]

Conscientiousness and its subfacets are related to multiple learning-related outcomes. MacCann, Duckworth and Roberts (2009) developed a questionnaire measuring seven subfacets of conscientiousness with 59 items. However, the resources required to complete such long scales often renders it unsuitable for large-scale educational assessment. Consequently, an economic and psychometrically sound conscientiousness questionnaire that is specifically customized for this context is needed. We developed and validated a short version of the MacCann et al. (2009) questionnaire. In study 1, French and German adaptations of the questionnaire were administered to a representative dataset comprising all ninth-graders in Luxembourg (N1=6325, Cohort 2017). Using an exhaustive search algorithm, we identified the optimal combination of four items for each subfacet by simultaneously considering three criteria: goodness of fit, factor saturation, and scalar measurement invariance across the German and French versions. In study 2, we validated our short 28-item questionnaire on a second, independent sample comprising 6,279 Luxembourgish ninth-graders (Cohort 2018). A 7-factor model assuming separate factors for each subfacet obtained acceptable fit (CFI=.93, RMSEA=.04, SRMR=.06). The criterion validity for each subfacet was tested by examining the relation to standardized achievement tests (SATs). In study 3, drawing on a dataset of 275 tenth-graders (linked longitudinally with the ninth-grade data from study 1), evidence of predictive validity (i.e., school grades) was examined. The subfacets of industriousness, caution and perfectionism showed the strongest relations with both SATs (study 2) and school grades (study 3). Our study delivered a short, valid and reliable questionnaire for the assessment of seven conscientiousness facets in the educational context. The scale is invariant across the German and French language versions and its brevity makes it suitable for large-scale educational assessment. [less ▲]

The stability of academic self-concept profiles – a latent transition analysis

Poster (2019, November 06)

Die Stabilität von akademischen Selbstkonzept-Profilen: Befunde einer „latent transition analysis“

Scientific Conference (2019, September)

Nach dem „internal-external frame of reference“ Modell (Marsh, 1986), spielen u.a. dimensionale Vergleiche (Vergleich der eigenen Leistung in einem Fach mit der eigenen Leistung in einem anderen Fach ... [more ▼]

Nach dem „internal-external frame of reference“ Modell (Marsh, 1986), spielen u.a. dimensionale Vergleiche (Vergleich der eigenen Leistung in einem Fach mit der eigenen Leistung in einem anderen Fach) eine Rolle für die Ausbildung fachspezifischer akademischer Selbstkonzepte (ASKs). Dabei kommt es zu Kontrasteffekten: Eine gute Leistung in Mathematik stärkt das Mathematik-ASK, schwächt aber das Deutsch-ASK und vice versa. Schüler schätzen sich daher oft als fähiger in einem der beiden Fächer ein. Diese Typisierung ließ sich auch in personenzentrierten Ansätzen zeigen. Eine Profilanalyse von Marsh, Lüdtke, Trautwein und Morin (2009) ergab, dass ASK-Profile entweder einem Mathematik-Typ (höheres Mathe-ASK, niedrigeres Sprach-ASK), einem Sprach-Typ (niedrigeres Mathe-ASK, höheres Sprach-ASK) oder einem Mischtyp (gleich hohe ASKs in allen Domänen) folgten. Ob diese Profile über die Zeit hinweg stabil sind oder Schüler in Abhängigkeit ihrer Erfahrungen im Schulalltag zwischen den Typen wechseln können, ist bisher unklar. Ziel der vorliegenden Studie ist es, diese Frage zu beantworten. Eine Stichprobe von N = 382 Schülern aus 18 Klassen und vier Bundesländern beantwortete einen Fragebogen zum Mathematik-, Physik-, Deutsch- und Englisch-ASK an zwei Messzeitpunkten, mit einem Intervall von vier Wochen. Eine „latent transition analysis“ ergab eine Vier-State Lösung (BIC = 7797.8, Entropie = 0.884). Die vier States beschrieben einen Mathematik-Typ (höhere ASKs in Mathematik und Physik, niedrigere ASKs in Deutsch und Englisch), einen Sprach-Typ (höhere ASKs in Deutsch und Englisch, niedrigere ASKs in Mathematik und Physik), und zwei Mischtypen (gleich hohe ASKs für alle Fächer mit generell höheren ASKs bzw. niedrigeren ASKs). Die Ergebnisse zeigten eine hohe Stabilität der States: Alle Personen wurden an beiden Messzeitpunkten demselben State zugewiesen; die Transitionswahrscheinlichkeiten lagen bei 0% bis 5%. Weitere Ergebnisse zur Stabilität von ASK-Profilen bei noch kürzeren Zeitabständen, basierend auf „Experience Sampling“ Daten, werden diskutiert. Marsh, H.W. (1986). Verbal and math self-concepts: An internal/external frame of reference model. American Educational Research Journal, 23, 129-149. Marsh, H.W., Lüdtke, O., Trautwein, U., & Morin, A.J.S. (2009). Classical latent profile analysis of academic self-concept dimensions: Synergy of person- and variable-centered approaches to theoretical models of self-concept. Structural Equation Modeling, 16, 191–225. [less ▲]

Developing and validating a short-form questionnaire for the assessment of seven conscientiousness facets in educational large-scale assessments

Scientific Conference (2018, November 09)

The influence of personality on academic outcomes has been demonstrated consistently in previous research. In particular, conscientiousness and its lower order facets generally show the strongest ... [more ▼]

The influence of personality on academic outcomes has been demonstrated consistently in previous research. In particular, conscientiousness and its lower order facets generally show the strongest predictive validity for academic success. MacCann et al. (2009) constructed a 68-item questionnaire for the comprehensive assessment of different conscientiousness facets in secondary education. However, such questionnaires are arguably too long for the use in large-scale educational assessments. Investigating the influence of conscientiousness facets within large-scale educational assessments is, nevertheless, crucial for advancing our knowledge of the differential influence of facets on various academic outcomes. Short and psychometrically sound questionnaires are thus required. Currently, no short and yet comprehensive questionnaire assessing the lower order facets of conscientiousness, that is suitable for large-scale educational assessments exists. Therefore, within the present investigation we develop a short-form instrument based upon the seven-factor version (59 items) of the questionnaire presented by MacCann et al. (2009), by using an exhaustive search algorithm and traditional item selection methods. Our sample consists of a large and representative dataset comprising all 9th grade students in Luxembourg from the Luxembourgish national school monitoring system (N = 6325; see epstan.lu), who answered French and German adaptations of the original 59-item questionnaire. We specified the exhaustive search algorithm to select the best possible combination of four items for each lower order facet, by considering goodness of fit criteria, factor saturation statistics, and measurement invariance between the German and French version. In addition, we used Mokken scale analysis and assessed the congruence of item wordings and theoretical definitions of each lower order facet to ensure content validity. We finally selected four to five items per scale based on our analyses. We found good fit statistics for all lower order facet scales (CFI >.95, RMSEA < 0.05), and acceptable to good factor saturation statistics for all but one of these scales (McDonalds ω > 0.7). On top of that, all scales show either scalar invariance, or partial scalar invariance between the German and French language version. The result of our investigation is a short and psychometrically sound instrument for the assessment of the lower order facets of conscientiousness. It is specifically tailored towards the unique context of the multilingual Luxembourgish educational system, and can be used in the upcoming Épstan cohorts. [less ▲]