Mixed Rasch modeling in assessing reading comprehension

in Aryadoust, Vahid; Raquel, Michelle (Eds.) Quantitative Data Analysis for Language Assessment (2019)

The nature of faking. A homogeneous and predictable construct?

in Psychological Assessment (2019), 31

A need for cognition scale for children and adolescents: Structural analysis and measurement invariance

in European Journal of Psychological Assessment (2019), 35

Need for Cognition (NFC) signifies “the tendency for an individual to engage in and enjoy thinking” (Cacioppo & Petty, 1982, p. 116). Up to now, no scale of sufficient psychometric quality existed to ... [more ▼]

Need for Cognition (NFC) signifies “the tendency for an individual to engage in and enjoy thinking” (Cacioppo & Petty, 1982, p. 116). Up to now, no scale of sufficient psychometric quality existed to assess NFC in children. Using data from three independent, diverse cross-sectional samples from Germany, Luxembourg, and Finland, we examined the psychometric properties of a new NFC scale intended to fill in this gap. In all samples, across grades levels ranging from 1 to 9, confirmatory factor analysis confirmed the hypothesized nested factor structure based on Mussel’s (2013) Intellect model, with one general factor Think influencing all items and two specific factors Seek and Conquer each influencing a subset of items. At least partial scalar measurement invariance with regard to grade level and sex could be demonstrated. The scale exhibited good psychometric properties and showed convergent and discriminant validity with an established NFC scale and other non-cognitive traits such as academic self-concept and interests. It incrementally predicted mostly statistically significant but relatively small portions of academic achievement variance over and above academic self-concept and interest. Implications for research on the development of NFC and its role as an investment trait in intellectual development are discussed. [less ▲]

Developing and validating a short-form questionnaire for the assessment of seven conscientiousness facets in educational large-scale assessments

Scientific Conference (2018, November 09)

The influence of personality on academic outcomes has been demonstrated consistently in previous research. In particular, conscientiousness and its lower order facets generally show the strongest ... [more ▼]

The influence of personality on academic outcomes has been demonstrated consistently in previous research. In particular, conscientiousness and its lower order facets generally show the strongest predictive validity for academic success. MacCann et al. (2009) constructed a 68-item questionnaire for the comprehensive assessment of different conscientiousness facets in secondary education. However, such questionnaires are arguably too long for the use in large-scale educational assessments. Investigating the influence of conscientiousness facets within large-scale educational assessments is, nevertheless, crucial for advancing our knowledge of the differential influence of facets on various academic outcomes. Short and psychometrically sound questionnaires are thus required. Currently, no short and yet comprehensive questionnaire assessing the lower order facets of conscientiousness, that is suitable for large-scale educational assessments exists. Therefore, within the present investigation we develop a short-form instrument based upon the seven-factor version (59 items) of the questionnaire presented by MacCann et al. (2009), by using an exhaustive search algorithm and traditional item selection methods. Our sample consists of a large and representative dataset comprising all 9th grade students in Luxembourg from the Luxembourgish national school monitoring system (N = 6325; see epstan.lu), who answered French and German adaptations of the original 59-item questionnaire. We specified the exhaustive search algorithm to select the best possible combination of four items for each lower order facet, by considering goodness of fit criteria, factor saturation statistics, and measurement invariance between the German and French version. In addition, we used Mokken scale analysis and assessed the congruence of item wordings and theoretical definitions of each lower order facet to ensure content validity. We finally selected four to five items per scale based on our analyses. We found good fit statistics for all lower order facet scales (CFI >.95, RMSEA < 0.05), and acceptable to good factor saturation statistics for all but one of these scales (McDonalds ω > 0.7). On top of that, all scales show either scalar invariance, or partial scalar invariance between the German and French language version. The result of our investigation is a short and psychometrically sound instrument for the assessment of the lower order facets of conscientiousness. It is specifically tailored towards the unique context of the multilingual Luxembourgish educational system, and can be used in the upcoming Épstan cohorts. [less ▲]

Dimensional and Social Comparisons Effects on Domain-Specific Self-Concepts and Interests: A Study of Elementary School Children from Luxembourg Across Two Waves

Scientific Conference (2018, November 09)

Already at the beginning of elementary school, students start to develop self-beliefs and attitudes that reflect their motivation, thoughts, and feelings about a specific school domain. The internal ... [more ▼]

Already at the beginning of elementary school, students start to develop self-beliefs and attitudes that reflect their motivation, thoughts, and feelings about a specific school domain. The internal/external frame of reference (I/E) model (Marsh, 1986) encapsulates this phenomenon by explicating the formation of academic self-concept through a combination of social (i.e. comparing one’s achievement in one domain with the achievement of one’s peers in the same domain) and dimensional (i.e. comparing one’s achievement in one domain with one’s achievement in another domain) comparison processes. The recently established generalized internal/external frame of reference (GI/E) model (Möller, Müller-Kalthoff, Helm, Nagy, & Marsh, 2015) builds on the I/E model by extending it to the formation of other subject-specific academic self-beliefs and attitudes, such as interest and test anxiety. Despite the salience of the (G)I/E model in educational research, studies investigating the formation of self-beliefs and attitudes according to this model remains scarce among elementary school children. This study aims to contribute to the current literature by examining the associations between verbal and mathematics achievement, on the one hand, and corresponding domain-specific self-concepts and interests, on the other hand. A population of Luxembourgish elementary school students was assessed twice, two years apart (once in Grade 1 and once in Grade 3) as part of the Luxembourgish school monitoring system (EpStan, cohorts 2014 & 2016; epstan.lu). Using a domain-specific approach, students were assessed on math and verbal achievement as well as German- and math self-concept and interest. This culminated in a sample of N = 3606 elementary school children who participated in both waves. German- and math self-concepts and interests were self-reported whereas standardized achievement tests (see epstan.lu) were used as indicators of math and verbal (Luxembourgish for Grade 1 and German for Grade 3) achievement. Cross-lagged structural equation modelling was performed in Mplus 8 using WLSMV estimation for categorical variables. Overall, the results suggest strong support for the (G)I/E model for Grade 3, while only partial support was found for the (G)I/E model for Grade 1. More specifically, for both Grade 1 and Grade 3, achievement was positively related to self-concept and interest within the same domain. Negative relations between achievement and self-concept and interest across domains were found in Grade 3, but not in Grade 1. The findings are discussed within the context of theory and research on self-concept differentiation processes, dimensional comparison theory, and the multilingual Luxembourgish educational system. [less ▲]

Using Log-File Data to Uncover Strategy Use in Complex Problem Solving

Scientific Conference (2018, September 18)

The aim of the current project is to utilize log-file data to enhance the understanding of strategy use in Complex Problem Solving (CPS). CPS can be defined as the ability to manipulate the existing ... [more ▼]

The aim of the current project is to utilize log-file data to enhance the understanding of strategy use in Complex Problem Solving (CPS). CPS can be defined as the ability to manipulate the existing variables of a particular novel, complex, intransparent, and dynamic environment successfully in order to reach a predefined goal. In order to successfully solve a CPS task, the goal-directed application and variation of certain strategies is necessary. To uncover the systematic use of these strategies, log-files have been deemed a fruitful resource as they contain not only the final results of a computer-based CPS item, but also the individual steps undertaken while solving such a task. Recent studies using log-files have highlighted the importance of some strategies for CPS, like the varying one variable at a time (VOTAT) strategy, or of engaging in noninterfering observations (i.e., idle rounds). In addition, other strategies have also been shown to be relevant in neighboring fields of CPS. However, comprehensive studies investigating a broad repertoire of strategies applied in CPS are scarce. Hence, on the basis of existing large-scale assessment data sets, the present project set out to investigate which strategies hidden in log-files are relevant for solving a CPS task in order to gain a more thorough understanding of how CPS tasks are being approached. Preliminary findings indicate that, in addition to VOTAT and idle rounds, particularly the flexibility of switching between strategies should be taken into account. [less ▲]

Dynamisches Problemlösen und figurale Matrizenaufgaben. Das MaFIN Item-Format

Poster (2018, September)

Die Vorhersage von Selbstkontrolle durch Need for Cognition. Generalisierbarkeit und Erklärungsansätze

Scientific Conference (2018, September)

Collaborative problem solving assessment in PISA 2015. Can computer agents replace humans? In F. Zehner & L. Khorramdel (Eds.), Collaborative problem solving. Standardized assessment at the large scale

Scientific Conference (2018, September)

Introducing MaFIN. A dynamic matrices finite state automata test

Scientific Conference (2018, July)