Do cognitive processes involved in solving reading comprehension items differ in students with differing language background?

One major global challenge of educational assessment that has to be addressed on a local level is the increasing number of students with immigration background usually speaking a different language at home compared to their native peers (OECD, 2012). Especially in large-scale contexts, however, individual and tailored testing responding to their specific (language) needs is not possible. Although DIF-analyses are common practice in current large-scale assessments, they only indicate whether and to what extent an item is biased but provide no information on which cognitive processes might cause that bias – crucial information when evaluating school systems. The current study goes beyond traditional DIF-analyses by using the IRT based linear logistic test model (LLTM; Fischer, 1973) that allows for modeling cognitive demands and therefore processes involved in each item. Specifically, we draw on a sample of more than 5000 Luxembourgish 3rd graders and analyze whether cognitive and linguistic item attributes (e.g., kind of inference that is needed to solve the item, textual coherence; Sonnleitner, 2008) of a large-scale reading comprehension test do possess different difficulty for students with varying language background. We do this by determining a cognitive model including such attributes that adequately describe item difficulty parameters in native students. Subsequently, we will cross validate this model in several sub-samples with varying language background. Results not only show if cognitive and linguistic item attributes do differ with regard to difficulty in the different samples but also if some cognitive processes do compensate each other in certain samples. It will be discussed how these results can be used to complement common DIF-analyses and to obtain more fine-grained information on students’ performance differences in reading comprehension.