Taking Language Out Of The Equation: The Assessment Of Basic Math Competence Without Language

Doctoral thesis (2020)

Although numeracy, next to literacy, is an essential skill in many knowledge based societies of the 21st century, between 5 and 10 % of the population suffer from more or less severe mathematics learning ... [more ▼]

Although numeracy, next to literacy, is an essential skill in many knowledge based societies of the 21st century, between 5 and 10 % of the population suffer from more or less severe mathematics learning disorders or dyscalculia. However, mathematical ability is not a pure construct. Instead, mathematical ability maintains a complex relationship with linguistic abilities. This relationship has significant implications for the assessment of a person’s mathematical ability in multilingual contexts. The hereby presented research project addresses the consequences of that relationship in the context of Luxembourg, a highly multilingual country at the center of Europe. The aim of the project was to tackle psychometric issues that arise when the test taker does not master the language of the test sufficiently and to offer an alternative solution to available assessment batteries based on verbal instructions and tasks. In the first study, we demonstrate the role of reading comprehension in the language of instruction on third grader’s performance in mathematics in Luxembourg and show that non-native speaker’s underachievement in mathematics can be largely or entirely explained by their lacking reading comprehension in the language of assessment. In the next study we report on the two first pilot studies with NUMTEST, an assessment battery that aims to measure children’s basic mathematical competence by replacing verbal instructions and task content with video instructions and animated tasks. The findings of these studies show that children’s basic mathematical competence can indeed be reliably assessed using this new paradigm. Opportunities and limitations of the paradigm are discussed. The third and final study of this project addresses the psychometric characteristics of this newly developed assessment battery. Its findings show that the NUMTEST battery provides for good reliability and concurrent validity all while being language neutral. In summary, the presented project provides for an encouraging proof of concept for the video instruction method while offering preliminary evidence for its validity as an early screener for math learning difficulties. [less ▲]

Taking Language out of the Equation: The Assessment of Basic Math Competence Without Language

in Frontiers in Psychology (2018)

While numerical skills are fundamental in modern societies, some estimated 5–7% of children suffer from mathematical learning difficulties (MLD) that need to be assessed early to ensure successful ... [more ▼]

While numerical skills are fundamental in modern societies, some estimated 5–7% of children suffer from mathematical learning difficulties (MLD) that need to be assessed early to ensure successful remediation. Universally employable diagnostic tools are yet lacking, as current test batteries for basic mathematics assessment are based on verbal instructions. However, prior research has shown that performance in mathematics assessment is often dependent on the testee’s proficiency in the language of instruction which might lead to unfair bias in test scores. Furthermore, language-dependent assessment tools produce results that are not easily comparable across countries. Here we present results of a study that aims to develop tasks allowing to test for basic math competence without relying on verbal instructions or task content. We implemented video and animation-based task instructions on touchscreen devices that require no verbal explanation. We administered these experimental tasks to two samples of children attending the first grade of primary school. One group completed the tasks with verbal instructions while another group received video instructions showing a person successfully completing the task.We assessed task comprehension and usability aspects both directly and indirectly. Our results suggest that the non-verbal instructions were generally well understood as the absence of explicit verbal instructions did not influence task performance. Thus we found that it is possible to assess basic math competence without verbal instructions. It also appeared that in some cases a single word in a verbal instruction can lead to the failure of a task that is successfully completed with non-verbal instruction. However, special care must be taken during task design because on rare occasions non-verbal video instructions fail to convey task instructions as clearly as spoken language and thus the latter do not provide a panacea to non-verbal assessment. Nevertheless, our findings provide an encouraging proof of concept for the further development of non-verbal assessment tools for basic math competence. [less ▲]

Project NUMTEST: Assessing basic number competence without language

Poster (2017, May 31)

While numerical skills are fundamental in modern societies, some estimated 5-7% of children suffer from a mathematical learning disorder, called developmental dyscalculia (DD). Nevertheless, universally ... [more ▼]

While numerical skills are fundamental in modern societies, some estimated 5-7% of children suffer from a mathematical learning disorder, called developmental dyscalculia (DD). Nevertheless, universally valid diagnostic instruments are still lacking, as all current DD test batteries are based on language instructions. Consequently, their measurements are tightly linked to the specific language context of test administration and thus their results cannot easily be compared across countries. Here we are showing results of the first pilot study of a research project that aims to develop a test for basic math abilities that does not rely on language instruction and minimizes language use. To this aim, video and animation based instructions were implemented on touchscreen devices. A first version of the tasks has been tested with two samples of first grade children in Luxembourg’s fundamental schools, of which half completed the same tasks with traditional verbal instructions. Our results indicate that performance in the experimental group was similar or better than the control group using verbal instructions. Relationships between linguistic background and the sample’s performance on one hand and qualitative usability aspects of nonverbal task instruction and tablet-pc use with young children will be discussed. [less ▲]