Intelligence as assessed by modern computer-based problem solving scenarios

Computer-based problem solving scenarios—so-called microworlds—have often been suggested as promising alternative assessment instruments of intelligence. Potential benefits compared to traditional paper-pencil tests involve tracking of students’ mental representations of the problems as well as their problem solving strategies by means of behavioral data. Though, it is still topic of ongoing debate whether the skills assessed by such microworlds are distinct from or identical to the construct of intelligence as measured by conventional reasoning tests. To address this issue, we thoroughly investigated construct and incremental validity of a recently developed microworld, the Genetics Lab (Sonnleitner et. al, 2011). We obtained data from a multilingual and representative Luxembourgish student sample (N = 563) who completed the Genetics Lab and 3 reasoning scales of an established intelligence test battery. Results of a Confirmatory Factor Analysis suggest that the construct assessed by the Genetics Lab is largely identical to the construct of intelligence as measured by traditional reasoning scales. Incremental validity was found with respect to the performance in a national assessment of students’ competencies and performance in the PISA study of 2009. Thus, the notion of microworlds to be a valuable measure of intelligence is supported.