Effect Size

What Is Effect Size?

Effect size measures how large an effect is — not just whether it's statistically significant. While a p-value tells you if a result is unlikely due to chance, effect size tells you if the result matters in practice.

Think of it this way: with a large enough sample, even a trivially small difference becomes statistically significant. Effect size cuts through the noise and answers the question every researcher actually cares about: "How big is the effect?"

Statistical significance tells you something happened. Effect size tells you whether anyone should care.

Why Effect Size Matters

1. APA requires it. The APA Publication Manual (7th ed.) requires effect sizes for all primary outcomes.
2. p-values are not enough. A p-value of .001 doesn't mean the effect is large — it means it's unlikely due to chance alone.
3. Comparability. Effect sizes let you compare results across different studies, different measures, and different sample sizes.
4. Power analysis. You need an expected effect size to calculate the sample size required for your study.

Common Effect Size Measures

Cohen's d

The most widely used effect size for comparing two group means. It expresses the difference between groups in standard deviation units.

$$d = \frac{\bar{X}_1 - \bar{X}_2}{s_p}$$

Where $\bar{X}_1$ and $\bar{X}_2$ are the group means, and $s_p$ is the pooled standard deviation.

Interpretation benchmarks (Cohen, 1988):

Hedges' g

Similar to Cohen's d but includes a correction for small sample bias. Preferred when either group has fewer than 20 participants.

$$g = d \times \left(1 - \frac{3}{4(n_1 + n_2) - 9}\right)$$

Eta-squared ($\eta^2$)

Used with ANOVA to describe the proportion of variance in the dependent variable explained by the independent variable.

$$\eta^2 = \frac{SS_{between}}{SS_{total}}$$

Worked Example

Scenario: A researcher compares test scores between two teaching methods. Group A (lecture, $n = 30$) scored $M = 78, SD = 10$. Group B (active learning, $n = 32$) scored $M = 85, SD = 12$.

Step 1: Calculate the pooled standard deviation.

$$s_p = \sqrt{\frac{(n_1 - 1)s_1^2 + (n_2 - 1)s_2^2}{n_1 + n_2 - 2}} = \sqrt{\frac{(29)(100) + (31)(144)}{60}} = \sqrt{\frac{2900 + 4464}{60}} = \sqrt{122.73} = 11.08$$

Step 2: Calculate Cohen's d.

$$d = \frac{85 - 78}{11.08} = \frac{7}{11.08} = 0.63$$

Interpretation: The effect size is $d = 0.63$, which falls between Cohen's benchmarks for a medium (0.5) and large (0.8) effect. Students in the active learning group scored about two-thirds of a standard deviation higher than the lecture group — a meaningful, practically significant difference.

How to Report Effect Size in APA Format

For a t-test with Cohen's d:

An independent samples t-test revealed that students in the active learning condition ($M$ = 85, $SD$ = 12) scored significantly higher than those in the lecture condition ($M$ = 78, $SD$ = 10), $t$(60) = 2.54, $p$ = .014, $d$ = 0.63.

Common Mistakes

1. Reporting only p-values. A significant p-value without effect size is incomplete.
2. Using Cohen's benchmarks rigidly. Small/medium/large are rough guides. In some fields, $d = 0.3$ is meaningful; in others, $d = 0.8$ is unremarkable. Consider your field's norms.
3. Forgetting Hedges' g for small samples. Cohen's d overestimates the effect in small samples ($n < 20$ per group).
4. Confusing $\eta^2$ with partial $\eta^2$. SPSS reports partial $\eta^2$ by default, which is always larger than $\eta^2$. Know which one you're reporting.

When to Use Each Measure

Related Concepts