Statistical Test Selection Flowchart

Use this table to find the right statistical test. Start with your research goal in the left column, then match your data type and study design to arrive at the recommended test.

Comparing Groups

Research Goal DV Data Type Design Recommended Test Non-Parametric Alternative
Compare 2 group means Continuous Between-subjects (independent groups) Independent-samples t-test Mann-Whitney U
Compare 2 related means Continuous Within-subjects (repeated measures / matched pairs) Paired-samples t-test Wilcoxon signed-rank
Compare 3+ group means Continuous Between-subjects, 1 IV One-way ANOVA Kruskal-Wallis H
Compare 3+ related means Continuous Within-subjects, 1 IV Repeated-measures ANOVA Friedman test
Compare means across 2+ IVs Continuous Between-subjects, 2+ IVs Factorial ANOVA Aligned rank transform

Associations & Relationships

Research Goal Variable Types Design Recommended Test Non-Parametric Alternative
Strength of linear relationship 2 continuous Bivariate Pearson r Spearman ρ / Kendall τ
Association between categories 2 categorical Cross-tabulation Chi-square test of independence Fisher's exact test (small n)
Goodness of fit to expected proportions 1 categorical Observed vs. expected Chi-square goodness-of-fit Binomial test (2 categories)

Prediction

Research Goal DV Type Predictors Recommended Test Notes
Predict a continuous outcome Continuous 1 continuous Simple linear regression Check linearity, normality of residuals
Predict a continuous outcome Continuous 2+ continuous/categorical Multiple linear regression Check multicollinearity (VIF)
Predict group membership Binary categorical 1+ continuous/categorical Logistic regression Reports odds ratios; no normality assumption for DV

Reliability & Agreement

Research Goal Data Type Design Recommended Measure Notes
Internal consistency of a scale Continuous / Likert items Multi-item measure Cronbach's α α ≥ .70 generally acceptable; use McDonald's ω for tau-nonequivalent items
Agreement between raters Continuous 2+ raters scoring same targets Intraclass Correlation (ICC) Choose ICC form based on rater model (one-way, two-way random, two-way mixed)
Agreement between raters Categorical 2 raters, nominal categories Cohen's κ Corrects for chance agreement; use Fleiss' κ for 3+ raters

Quick Decision Rules

  • 2 groups → t-test. 3 or more groups → ANOVA.
  • Same participants measured twice → paired / repeated-measures. Different participants → independent / between-subjects.
  • DV is continuous → parametric tests. DV is ordinal or non-normal → non-parametric alternatives.
  • DV is categorical → chi-square or logistic regression.
  • Want to predict an outcome → regression. Want to describe a relationship → correlation.
  • Assumptions violated? Try a transformation first, then a non-parametric test, then bootstrapping.