The development and validation of a short conscientiousness questionnaire for large-scale educational assessment

Conscientiousness and its subfacets are related to multiple learning-related outcomes. MacCann, Duckworth and Roberts (2009) developed a questionnaire measuring seven subfacets of conscientiousness with 59 items. However, the resources required to complete such long scales often renders it unsuitable for large-scale educational assessment. Consequently, an economic and psychometrically sound conscientiousness questionnaire that is specifically customized for this context is needed.
We developed and validated a short version of the MacCann et al. (2009) questionnaire. In study 1, French and German adaptations of the questionnaire were administered to a representative dataset comprising all ninth-graders in Luxembourg (N1=6325, Cohort 2017). Using an exhaustive search algorithm, we identified the optimal combination of four items for each subfacet by simultaneously considering three criteria: goodness of fit, factor saturation, and scalar measurement invariance across the German and French versions. In study 2, we validated our short 28-item questionnaire on a second, independent sample comprising 6,279 Luxembourgish ninth-graders (Cohort 2018). A 7-factor model assuming separate factors for each subfacet obtained acceptable fit (CFI=.93, RMSEA=.04, SRMR=.06). The criterion validity for each subfacet was tested by examining the relation to standardized achievement tests (SATs). In study 3, drawing on a dataset of 275 tenth-graders (linked longitudinally with the ninth-grade data from study 1), evidence of predictive validity (i.e., school grades) was examined. The subfacets of industriousness, caution and perfectionism showed the strongest relations with both SATs (study 2) and school grades (study 3). Our study delivered a short, valid and reliable questionnaire for the assessment of seven conscientiousness facets in the educational context. The scale is invariant across the German and French language versions and its brevity makes it suitable for large-scale educational assessment.