Construct validity of complex problem solving: A comprehensive view on different facets of intelligence and school grades

in Intelligence (2016), 54

Although Complex Problem Solving (CPS) has attracted increasing amounts of attention in recent years (e.g., PISA study), the role of CPS in the nomological network of intelligence is controversial. The ... [more ▼]

Although Complex Problem Solving (CPS) has attracted increasing amounts of attention in recent years (e.g., PISA study), the role of CPS in the nomological network of intelligence is controversial. The question of whether CPS is a distinct construct is as old as CPS research itself, but previous studies have had specific shortcomings when addressing the question of whether CPS is a separable or independent construct. The aim of the present study was, therefore, to combine the advantages of previous studies to facilitate a less biased view of the relation between CPS and established intelligence constructs. A sample of 227 German university students worked on a comprehensive measure of intelligence (Berlin Intelligence Structure test) and two CPS assessment tools (MicroDYN and MicroFIN). Furthermore, final school grades (GPA) served as an external criterion. We applied confirmatory factor analyses and structural equation modeling to investigate the relation between CPS and established intelligence constructs on the basis of different psychometric approaches (i.e., first-order model, nested factor model). Moreover, we examined the incremental validity of CPS in explaining GPA beyond established intelligence constructs. Results indicate that CPS represents unique variance that is not accounted for by established intelligence constructs. The incremental validity of CPS was found only when a commonly used narrow operationalization of intelligence was applied (i.e., figural reasoning) but not when a broad operationalization was applied. [less ▲]

Determinants of individual occupational careers in the 21st century. Does complex problem solving matter beyond general mental ability?

Scientific Conference (2015, May 21)

Purpose Complex problem solving (CPS) describes the interaction with dynamic and nonroutine tasks, and has been included in PISA 2012 as a factor for employability. This study examines whether CPS can ... [more ▼]

Purpose Complex problem solving (CPS) describes the interaction with dynamic and nonroutine tasks, and has been included in PISA 2012 as a factor for employability. This study examines whether CPS can also contribute to the prediction of career advancement in jobs beyond general mental ability (GMA) as one of the best predictors. Design/Methodology Using latent structural equation modeling (SEM), we analyzed a sample of technicians, service/trade workers, and assemblers (n=245) at a German automotive company. A computer-based assessment measured participants' CPS and GMA levels. The dependent variables were the participants’ job level (ISCO-08) and professional training days. Results CPS and GMA both correlated significantly with career advancement (from .18 to .26, all p < .01). The models showed good fit and indicated that CPS explained incremental variance in one of two indicators (β = .14 for trainings, p < .05; ΔR2 = .02) in comparison with GMA alone (β = .24, p < 0.01; R2= .06). Limitations Analyses did not include processes information from CPS assessment as potential advantage. The company-based sample and cross-sectional data restrict inferences. Research Implications Our findings suggest positive relations between CPS and career advancement even when controlling for GMA. Hence, CPS could be a valuable addition for the study of careers and personnel selection test batteries. Originality/Value The first evaluation of CPS in career research gave a general indication of an as-yet-to-be-defined role of CPS, especially when considering the task characteristics compared to complex and demanding jobs, and process data available through CPS. [less ▲]

Log file analyses in educational large-scale assessments: A new window to students’ minds?

Presentation (2015, March)

The Assessment of 21st Century Skills in Industrial and Organizational Psychology: Complex and Collaborative Problem Solving

in Industrial and Organizational Psychology (2015), 8(2), 1-31

In the current paper, we highlight why and how industrial and organizational psychology can take advantage of research on 21st century skills and their assessment. We present vital theoretical ... [more ▼]

In the current paper, we highlight why and how industrial and organizational psychology can take advantage of research on 21st century skills and their assessment. We present vital theoretical perspectives, a suitable framework for assessment, and exemplary instruments with a focus on advances in the assessment of Human Capital. Specifically, Complex Problem Solving (CPS) and Collaborative Problem Solving (ColPS) are two transversal skills (i.e., skills that span multiple domains) that are generally considered critical in the 21st century workplace. The assessment of these skills in education has linked fundamental research with practical applicability and has provided a useful template for workplace assessment. Both CPS and ColPS capture the interaction of individuals with problems that require the active acquisition and application of knowledge in individual or group settings. To ignite a discussion in industrial and organizational psychology, we discuss advances in the assessment of CPS and ColPS and propose ways to move beyond the current state of the art in assessing job-related skills. [less ▲]

Extending the assessment of Complex Problem Solving to finite state automata. Embracing heterogeneity

in European Journal of Psychological Assessment (2015), 31

Linking complex problem solving to Innovation via opportunity competence

in International Journal of Lifelong Education (2015), 34

Extending the Assessment of Complex Problem Solving to Finite State Automata Embracing Heterogeneity

Scientific Conference (2014, October)

Enhancing complex problem solving skills in the lifelong learning context. Evidence based on microdata. Paper presented at the 6th International Symposium on Human Capital in Beijing,

Scientific Conference (2014, October)

Komplexes Problemlösen, schulfachliche Kompetenzen und ihre Relation zu Schulnoten

in Zeitschrift für Pädagogische Psychologie (2014), 28(4), 205215

The importance of Complex Problem Solving (CPS) within the educational context is well established. This is one of the reasons why CPS plays a prominent role in educational large-scale assessments (e.g ... [more ▼]

The importance of Complex Problem Solving (CPS) within the educational context is well established. This is one of the reasons why CPS plays a prominent role in educational large-scale assessments (e.g., PISA) besides school competencies. However, recent research on CPS and its connection to school performance did not include such school competencies as such, which have proven to be strong predictors of school grades. Consequently, the aim of this study is to close this gap and to examine the relation between CPS and competencies in mathematics and reading. Based on a sample of N=1908 Finish high school students, structural equation modeling was used to analyse the relation of CPS, school competencies, and school grades. In general, the results showed an incremental predictive power of CPS over and above school competencies on school grades in mathematics and mother language. However, differential effects showed a higher importance of CPS in the mathematic domain in comparison to the language domain especially if controlled for reasoning. Implications for the construct of CPS and its importance within the educational context are discussed. [less ▲]