[en] This project develops methods and procedures to (a) quantify the quality of primary studies in meta-analyses; and (b) account for primary-study quality in moderator analyses. As part of the project, we develop an analytic procedure to create study quality indicators and incorporate them in the meta-analysis. We will present this procedure in a step-by-step tutorial with illustrative examples.
Research center :
- Faculty of Language and Literature, Humanities, Arts and Education (FLSHASE) > Luxembourg Centre for Educational Testing (LUCET)
Disciplines :
Mathematics Theoretical & cognitive psychology
Author, co-author :
Scherer, Ronny; University of Oslo, Norway > Faculty of Educational Sciences > Centre for Educational Measurement at the University of Oslo (CEMO)
Emslander, Valentin ; University of Luxembourg > Faculty of Humanities, Education and Social Sciences (FHSE) > LUCET
Research Questions
Which analytic steps and methodological decisions are relevant for the creation of quality indicators or primary studies in meta-analyses?
How can these indicators be incorporated in meta-analytic models?
Hypotheses
We hypothesize that the following steps and issues will be relevant:
(a) the level at which quality is measured and the resultant type of quality that is measured;
(b) the directions and correlations of quality indicators;
(c) the assumptions behind aggregated quality scores;
(d) the nature and scales of indicators;
(e) the choice for an adequate, meta-analytic baseline model;
(f) the study of possible non-linear effects.
We expect the following meta-analytic steps to gain insights into the effects of study quality:
(1) Moderation by study quality;
(2) Moderation after controlling for study quality;
(3) Moderation including interactions with study quality.
Data Description
In our illustrative examples, we will use one or more existing, meta-analytic datasets:
(a) Scherer et al. (2019), https://doi.org/10.1037/edu0000314
(b) Scherer et al. (2021), https://doi.org/10.31234/osf.io/bucf9
(c) Datasets published in the R package "metadat" (by Wolfgang Viechtbauer), https://cran.r-project.org/web/packages/metadat/readme/README.html
(d) Emslander & Scherer (2022), https://doi.org/10.17605/OSF.IO/VN5PE
