Pre-Test/Post-Test Designs: Which Statistical Test to Use

Pre-test/post-test designs are among the most common in educational and social science research. You measure participants before an intervention, deliver the intervention, then measure them again. Simple concept — but choosing the right statistical test is where things get confusing.

The right test depends on your specific design. Let's walk through the options.

Design 1: One Group, Pre and Post

Scenario: You give all participants a training program and measure their knowledge before and after.

Test: Paired samples t-test

This is the simplest pre-post design. You're comparing two related means (same people, two time points). The paired t-test accounts for the fact that each person's pre-score and post-score are linked.

In SPSS: Analyze → Compare Means → Paired-Samples T Test

Limitation: Without a control group, you can't rule out alternative explanations (maturation, testing effects, history). Any change might have happened without the intervention.

Design 2: Two Groups, Pre and Post

Scenario: A treatment group receives the intervention; a control group doesn't. Both are measured before and after.

Option A: Mixed ANOVA (Split-Plot ANOVA)

A mixed ANOVA has one between-subjects factor (group: treatment vs. control) and one within-subjects factor (time: pre vs. post). The key result is the interaction effect — it tells you whether the change over time differs between groups. That's usually your main research question.

In SPSS: Analyze → General Linear Model → Repeated Measures

Option B: ANCOVA

An analysis of covariance uses the post-test score as the dependent variable, group as the independent variable, and the pre-test score as a covariate. This approach statistically controls for initial differences between groups.

Many methodologists actually prefer ANCOVA over mixed ANOVA for pre-post designs because it has greater statistical power and handles pre-existing group differences more effectively.

In SPSS: Analyze → General Linear Model → Univariate (add pre-test as covariate)

Design 3: Multiple Groups, Pre and Post

Scenario: Three or more groups (e.g., two treatment conditions and a control) measured before and after.

Test: Mixed ANOVA with one between-subjects factor (group with 3+ levels) and one within-subjects factor (time). Follow up a significant interaction with simple effects or pairwise comparisons.

Alternatively, use ANCOVA with group as the factor and pre-test scores as the covariate.

Design 4: One Group, Multiple Time Points

Scenario: You measure the same participants at three or more time points (e.g., pre, mid, post, follow-up).

Test: Repeated measures ANOVA with one within-subjects factor (time with 3+ levels). If the overall test is significant, use post-hoc pairwise comparisons to determine which time points differ.

Important: Check Mauchly's test of sphericity. If violated (which is common with 3+ time points), use the Greenhouse-Geisser correction.

Which Approach Should You Choose?

Here's a quick decision guide:

Don't Forget These Steps

1. Check assumptions. Normality, homogeneity of variance, and sphericity (for repeated measures) all matter.
2. Report effect sizes. Partial eta-squared for ANOVA, Cohen's d for pairwise comparisons.
3. Interpret the interaction. In a two-group pre-post design, the interaction tells the real story. Don't just report main effects.
4. Create a clear results table showing means and standard deviations at each time point for each group.

A Common Mistake

Many students analyze pre-post data by computing gain scores (post minus pre) and running an independent t-test on the gains. While this isn't always wrong, ANCOVA is generally preferred because it has better statistical power and controls for regression to the mean. If you've been told to use gain scores, check with your methodology advisor — ANCOVA may be a better option.