TESTING 21ST CENTURY SKILLS IN A CHANGING NATURE OF WORK: THE CONSTRUCT VALIDITY OF COMPLEX PROBLEM SOLVING AND ORGANIZATIONAL LEARNING

Doctoral thesis (2017)

This thesis investigates the empirical assessment and construct validity of complex problem-solving (CPS) skills and organizational learning (OL) in a changing nature of work. Technological and ... [more ▼]

This thesis investigates the empirical assessment and construct validity of complex problem-solving (CPS) skills and organizational learning (OL) in a changing nature of work. Technological and organizational change enlarge the gap between the type of employee required by the market and the employees who are actually being provided by higher education as human capital to the market. Two constructs, CPS and OL, might strengthen human capital to close the emerging gap in a changing nature of work. [less ▲]

Determinants of individual occupational careers in the 21st century. Does complex problem solving matter beyond general mental ability?

Scientific Conference (2015, May 21)

Purpose Complex problem solving (CPS) describes the interaction with dynamic and nonroutine tasks, and has been included in PISA 2012 as a factor for employability. This study examines whether CPS can ... [more ▼]

Purpose Complex problem solving (CPS) describes the interaction with dynamic and nonroutine tasks, and has been included in PISA 2012 as a factor for employability. This study examines whether CPS can also contribute to the prediction of career advancement in jobs beyond general mental ability (GMA) as one of the best predictors. Design/Methodology Using latent structural equation modeling (SEM), we analyzed a sample of technicians, service/trade workers, and assemblers (n=245) at a German automotive company. A computer-based assessment measured participants' CPS and GMA levels. The dependent variables were the participants’ job level (ISCO-08) and professional training days. Results CPS and GMA both correlated significantly with career advancement (from .18 to .26, all p < .01). The models showed good fit and indicated that CPS explained incremental variance in one of two indicators (β = .14 for trainings, p < .05; ΔR2 = .02) in comparison with GMA alone (β = .24, p < 0.01; R2= .06). Limitations Analyses did not include processes information from CPS assessment as potential advantage. The company-based sample and cross-sectional data restrict inferences. Research Implications Our findings suggest positive relations between CPS and career advancement even when controlling for GMA. Hence, CPS could be a valuable addition for the study of careers and personnel selection test batteries. Originality/Value The first evaluation of CPS in career research gave a general indication of an as-yet-to-be-defined role of CPS, especially when considering the task characteristics compared to complex and demanding jobs, and process data available through CPS. [less ▲]

The Assessment of 21st Century Skills in Industrial and Organizational Psychology: Complex and Collaborative Problem Solving

in Industrial and Organizational Psychology (2015), 8(2), 1-31

In the current paper, we highlight why and how industrial and organizational psychology can take advantage of research on 21st century skills and their assessment. We present vital theoretical ... [more ▼]

In the current paper, we highlight why and how industrial and organizational psychology can take advantage of research on 21st century skills and their assessment. We present vital theoretical perspectives, a suitable framework for assessment, and exemplary instruments with a focus on advances in the assessment of Human Capital. Specifically, Complex Problem Solving (CPS) and Collaborative Problem Solving (ColPS) are two transversal skills (i.e., skills that span multiple domains) that are generally considered critical in the 21st century workplace. The assessment of these skills in education has linked fundamental research with practical applicability and has provided a useful template for workplace assessment. Both CPS and ColPS capture the interaction of individuals with problems that require the active acquisition and application of knowledge in individual or group settings. To ignite a discussion in industrial and organizational psychology, we discuss advances in the assessment of CPS and ColPS and propose ways to move beyond the current state of the art in assessing job-related skills. [less ▲]

Thematic workshop on complex problem solving.

Scientific Conference (2015)

Linkages of complex problem solving and general mental ability to career advancement. Does a transversal skill reveal incremental predictive validity?

in International Journal of Lifelong Education (2015), 34

Progress in the LLLight project. Report from the assessment team from Luxembourg.

Presentation (2014, October)

Determinants of Individual Occupational Careers in the 21st Century – Why Complex Problem Solving Matters Beyond General Mental Ability

Scientific Conference (2014, August 27)

The OECD (2010) views problem solving as essential for lifelong learning. Recently, the 2012 PISA cycle administered domain-general problems, which featured the complexity of the structure, dynamics and ... [more ▼]

The OECD (2010) views problem solving as essential for lifelong learning. Recently, the 2012 PISA cycle administered domain-general problems, which featured the complexity of the structure, dynamics and interconnections of variables, and intransparency of the situation. This emphasis on domain-general problems expresses a paradigmatic shift to lifelong learning. In an attempt towards an integrative process theory of CPS, Fischer, Greiff and Funke (2012) portray knowledge acquisition and knowledge application as the core processes of CPS, which are distinguishable from classical cognitive ability (Dörner, 1986). These core processes are assessed in research via innovative computer-based microworlds. These microworlds simulate complex problems, which share features with real world problems, require knowledge acquisition and knowledge application for a successful task completion and offer a wide range of tasks and approaches (Greiff, Holt, et al., 2013; Greiff, Fischer, et al., 2013). Initiatives looking for the facilitation of so-called 21st century skills (e.g., Griffin, McGaw, & Care, 2012; National Research Council, 2012a; OECD, 2013a, 2013b) strongly relate CPS to demands for professional development as resulting from shifts towards non-routine problem solving and communication tasks in the working lives of adults (e.g., successfully adapting towards IT-depending occupations; Autor, Levy, & Murnane, 2003). So far, the application of CPS in research and practice has mainly been restricted to primary and secondary education. Lifespan or educational research on the role of transversal cognitive skills such as CPS for learning later in life is scarce at best and adult education practice fails to fully tap into the requirements of a working reality (Smith & Reio, 2006), which is defined by uncertainties, contradictions, and dynamic changes. CPS is considered to be important for lifelong learning and occupational success, but underrepresented in research and practice. As a consequence, adult education’s mission to provide opportunities for career preparation (e.g., Gianakos, 1996) and intellectual growth (Perry, 1999) seem to be at risk. Hence, the research goals in this study are to (1) enlarge the understanding for determinants of occupational career indicators (e.g. lifelong learning efforts) and eventually (2) encourage the inclusion of CPS on the adult educational research and practice agenda. Our research goals are built on a rich research fundus of the impact of GMA on professional development (e.g., Schmidt & Hunter, 1998) and promising results on the predictive validity of CPS for academic achievement (e.g., Schweizer, Wüstenberg, & Greiff, 2013; Sonnleitner, Keller, Martin, & Brunner, 2013; Wüstenberg, Greiff, & Funke, 2012) and occupational performance (Abele et al., 2012; Danner et al., 2011). Hence, the targeted constructs are GMA, and CPS as a transversal cognitive skill in occupational careers and professional development. [less ▲]

Validitätsforschung in Large-Scale Assessments. Innovationen ohne Validitätsdilemma am Beispiel des Komplexen Problemlösens

Scientific Conference (2014, March)

Komplexes Problemlösen als eigenständiges Konstrukt

Scientific Conference (2013, September 25)

Recently published studies on Complex Problem Solving (CPS) suggest that assessments of CPS using multiple complex systems are only moderately related to tests of classical cognitive abilities. Further ... [more ▼]

Recently published studies on Complex Problem Solving (CPS) suggest that assessments of CPS using multiple complex systems are only moderately related to tests of classical cognitive abilities. Further, CPS assessments show incremental validity beyond tests of other cognitive abilities when predicting relevant outcomes. However, these empirical accounts have relied on single CPS assessment instruments. We do not know whether these findings will generalize to the construct level across different CPS assessment instruments. To answer this question, we tested a sample of N = 339 German university students who completed three CPS assessment instruments based on multiple complex systems (MicroDYN, the Genetics Lab, and MicroFIN) and the matrices subtest of the Intelligence Structure Test as measure of reasoning. Students further reported their school grades. Analyses including latent multitrait–multimethod models provided support for the conceptualization of CPS as a complex cognitive ability. Results indicated that different CPS assessment instruments showed sufficient convergent validity (with a consistency mostly between .50 and .60). In addition, we found evidence for the divergent validity of CPS from reasoning (reasoning predicted two CPS facets, knowledge and control, àKNOW = .49 and àCON = .53, respectively). In the prediction of academic achieve- ment, CPS explained variance in natural science grades after we controlled for reasoning (àCPS = .22), whereas social science grades were not predicted. Our findings suggest that the validity of CPS generalizes across different measurement instruments. [less ▲]