Comparing regression approaches in modelling (non-)compensatory judgement formation

Applied research on judgment formation, e.g. in education, is interested in identifying the underlying judgment rules from empirical judgment data. Psychological theories and empirical results on human judgment formation support the assumption of compensatory strategies, e.g. (weighted) linear models, as well as non compensatory (heuristic) strategies as underlying judgment rules. Previous research repeatedly demonstrated that linear regression models well fitted empirical judgment data, leading to the conclusion that the underlying cognitive judgment rules were also linear and compensatory. This simulation study investigated whether a good fit of a linear regression model is a valid indicator of a compensatory cognitive judgment formation process. Simulated judgment data sets with underlying compensatory and noncompensatory judgment rules were generated to reflect typical judgment data from applied educational research. Results indicated that linear regression models well fitted even judgment data with underlying non compensatory judgment rules, thus impairing the validity of the fit of the linear model as an indicator of compensatory cognitive judgment processes.