Using Complex Problem Solving Simulations for General Cognitive Ability Assessment: the Genetics Lab Framework

in International Journal of Information Science and Intelligent System (2013), 2(4), 71-88

This paper presents the conception and operation of “the Genetics Lab,” a research platform designed for general intelligence measurement. This tool has been developed and refined to measure cognitive ... [more ▼]

This paper presents the conception and operation of “the Genetics Lab,” a research platform designed for general intelligence measurement. This tool has been developed and refined to measure cognitive skills of students in Luxembourg, Thanks to its generic and clear architecture, “the Genetics Lab” makes a major contribution by addressing technical weaknesses and flaws met with tools previously used to assess intelligence. This new platform can be used as groundwork for other evaluation solutions. In this paper, we explore in depth the main fields of e-assessment such as instructions, localized and multilingual content production and display, traces, and scoring. We also describe how the platform was put into action to create complex simulations and to make data collection resulting from the user’s test a flawless process. This paper is a continuation and extension of earlier work. [less ▲]

User’s guide to apply, configure and adapt the Genetics Lab

E-print/Working paper (2012)

The Genetics Lab. Acceptance and psychometric characteristics of a computer-based microworld to assess Complex Problem Solving

in Psychological Test and Assessment Modeling (2012), 54

Computer-based problem solving scenarios or “microworlds” are contemporary assessment instruments frequently used to assess students’ complex problem solving behavior – a key aspect of today’s educational ... [more ▼]

Computer-based problem solving scenarios or “microworlds” are contemporary assessment instruments frequently used to assess students’ complex problem solving behavior – a key aspect of today’s educational curricula and assessment frameworks. Surprisingly, almost nothing is known about their (1) acceptance or (2) psychometric characteristics in student populations. This article introduces the Genetics Lab (GL), a newly developed microworld, and addresses this lack of empirical data in two studies. Findings from Study 1, with a sample of 61 ninth graders, show that acceptance of the GL was high and that the internal consistencies of the scores obtained were satisfactory. In addition, meaningful intercorrelations between the scores supported the instrument’s construct validity. Study 2 drew on data from 79 ninth graders in differing school types. Large to medium correlations with figural and numerical reasoning scores provided evidence for the instrument’s construct validity. In terms of external validity, substantial correlations were found between academic performance and scores on the GL, most of which were higher than those observed between academic performance and the reasoning scales administered. In sum, this research closes an important empirical gap by (1) proving acceptance of the GL and (2) demonstrating satisfactory psychometric properties of its scores in student populations. [less ▲]

Assessing Intelligence for Education in the 21st Century: The Benefits of Microworlds

Scientific Conference (2011)

Computer-based problem solving scenarios—so-called microworlds—are contemporary educational assessment instruments of intelligence that offer several benefits compared to traditional paper-pencil tests ... [more ▼]

Computer-based problem solving scenarios—so-called microworlds—are contemporary educational assessment instruments of intelligence that offer several benefits compared to traditional paper-pencil tests. This involves tracking of students’ mental representations of the problems as well as their problem solving strategies by means of behavioral data which provides key information for educational interventions. Moreover, microworlds realize game-like characteristics that may increase test motivation and reduce test anxiety. In the present study, the Genetics Lab, a newly developed microworld, was completed by a representative sample of more than 800 Luxembourgish students. Students chose among three different languages (German, French and English) in which the problem content of the Genetics Lab was presented. The present paper analyzes the psychometric properties of the various performance scores derived for the Genetics Lab with respect to their relations to school grades, and measurement invariance across gender, chosen test language, and migration background. Moreover, a direct comparison with traditional measures of intelligence demonstrated construct validity of the performance scores of the Genetics Lab. In sum, the results obtained for the Genetics Lab show the benefits of behavioral data obtained for computer-based problem-solving scenarios and support the notion of microworlds to be a valuable measure of intelligence. [less ▲]