Why Hedges’ g*s based on the non-pooled standard deviation should be reported with Welch’s t-test

<p>Researchers are generally required to report and interpret effect sizes and associated confidence intervals. When comparing two independent groups, the most commonly used estimator of effect size is Cohen’s ds where sample mean difference is divided by the pooled standard deviation. However, computing the pooled error term is not valid when both groups do not share common population variances. Furthermore, the assumption of equal population variances is unlikely in many psychological fields. Consequently, researchers shift to the use of Welch’s t-test over Student’s t-test in the context of hypothesis testing. Meanwhile, the question which effect size to report when equal variances are not assumed remains open. Based on Monte Carlo simulations, we compare Hedges’ gs (i.e. Cohen’s ds with correction for bias) to Glass’s gs, Shieh’s gs and Hedges’ g_s^*. Comparisons are made under normality as well as under realistic deviations from the assumptions of normality and equal variances. Although it is not directly related with Welch’s t-test (unlike Shieh’s gs), we recommend the use of Hedges’ g_s^* because it shows better properties than all other estimators. Practical recommendations, R package and Shiny App in order to compute effect size estimators and confidence intervals are provided.</p>