French speakers reading German and German speakers reading French: Do they take different tests?

The past two decades have seen an important number of studies evaluating reading comprehension. However, it has become increasingly difficult to compare results from different studies, deploying different tests, targeting different languages and persons with different language backgrounds. A better knowledge of the attributes that are responsible for the difficulty of reading comprehension items would hence lead to a more theory-driven test specification. The current study aims at exploring the benefits of using the linear logistic test model (LLTM; Fischer, 1973) regarding the identification of those cognitive, linguistic and test-specific attributes that best describe and explain reading test performance. Strongly based on scientific findings from reading literacy studies, a list of attributes (e.g., making inferences, position of correct option in MC item) was determined. The same list was used to specify both 33 German and 34 French reading comprehension items with known and adequate statistical parameters. This specification resulted in a Q-matrix, in which each attribute was given a weight, depending on whether the attribute was essential or not for solving the item. Finally, the LLTM was applied to the data of Luxembourgish/German, French, and Portuguese speaking 9th graders that had previously taken the German and the French reading test. The results from the LLTM modeling show a significant overlap between both tests, and between the three language groups regarding the strength of the attributes in explaining item difficulty. The findings help explain the construct of reading comprehension and are discussed regarding their implications on test construction and teaching.