Post-Hoc Tests After ANOVA (Tukey, Games-Howell, Dunn's)

The omnibus test and the post-hoc test answer different questions

A one-way ANOVA's F-test asks a single, narrow question: does at least one group mean differ from at least one other, somewhere among the groups you compared? A significant F confirms that something is going on, but the test itself is silent on which groups, by how much, or in which direction. Kruskal-Wallis, the rank-based analogue used when the outcome is skewed, ordinal, or otherwise unsuited to a mean, asks the same shape of question about the ranked data and is equally silent on where the difference sits.⁴ Reporting only the omnibus result, a single sentence noting that "groups differed significantly" with an F or H statistic and a P value, and stopping there is incomplete: it tells a reader that a difference exists without telling them which comparison drives it, which is exactly the follow-up question a reviewer or a treating clinician will ask next.

That follow-up is a genuinely second test, run on the same data, answering a genuinely different question, and it comes with its own assumptions that don't automatically inherit from the omnibus test you happened to run first.

Four ways to run the comparison, and why they aren't interchangeable

Once the omnibus result is significant, the choice of post-hoc procedure depends on two things: whether the comparisons are parametric or rank-based, and, for the parametric case, whether the groups have roughly equal variance. Four procedures cover almost everything a clinical or surgical dataset will need.

Tukey's HSD test compares every pair of group means using the studentized range distribution, which keeps the family-wise error rate at the nominal α across all pairwise comparisons at once rather than letting it compound the way a series of uncorrected t-tests would.² It assumes the variances are equal across groups and performs best with equal or near-equal group sizes; when either assumption breaks, particularly the variance assumption, its error-rate control stops being trustworthy.

Games-Howell exists for exactly that failure case. It uses a Welch-type correction, calculating a separate standard error and degrees of freedom for every pair rather than pooling variance across all groups, so it stays valid whether the groups have equal variances and sample sizes or not.³ The practical rule is simple: check Levene's test (or the omnibus test's own equal-variances diagnostic) first, and let its result decide between the two, not familiarity or habit.

Dunn's test is the post-hoc procedure built for Kruskal-Wallis. Because Kruskal-Wallis operates on ranks rather than raw values, its follow-up test also compares mean ranks between pairs of groups rather than means, and Dunn's original 1964 paper specifies the correction needed to keep the family-wise error rate controlled across every pairwise rank comparison.⁵ In practice, Dunn's test is almost always paired with a Bonferroni correction (or a close variant such as Holm's step-down method), applied across every pairwise comparison in the family, not just the ones that turn out to look interesting.

A plain Bonferroni-corrected set of pairwise comparisons, dividing α by the number of comparisons and applying that stricter threshold to each individual test, is the simplest of the four and remains defensible for a small, prespecified set of comparisons decided before the data were seen. It is conservative, loses power quickly as the number of comparisons grows, and isn't gated by an omnibus test the way Tukey, Games-Howell, and Dunn's test typically are.

A worked example: three surgical approaches, one skewed outcome

Suppose a study compares postoperative length of stay across three surgical approaches, open, laparoscopic, and robotic, in a cohort of 150 patients. Length of stay is bounded at zero and right-skewed by a small number of prolonged admissions, so the analysis plan calls for Kruskal-Wallis rather than one-way ANOVA. (This scenario is illustrative, not drawn from a specific published dataset.) The Kruskal-Wallis test returns H(2) = 9.8, P = .007, confirming that length of stay differs across at least one pair of approaches, but not which one.

Dunn's test with a Bonferroni correction across all three pairwise comparisons finds that the open-versus-robotic comparison is significant (adjusted P = .006) while open-versus-laparoscopic and laparoscopic-versus-robotic are not (adjusted P = .21 and .38). The Methods and Results sentences that follow from this, in AMA format, would read: "Length of stay differed significantly across surgical approach (Kruskal-Wallis H(2) = 9.8, P = .007). Dunn's post hoc test with Bonferroni correction showed a significant difference between the open and robotic groups (adjusted P = .006), with no significant difference between the laparoscopic group and either of the other two approaches." That is a materially different, more useful claim than "length of stay differed significantly by approach" on its own, and it is the sentence a reviewer expects to see once the omnibus test is significant.

How to run it — SPSS vs StatsPlease

In IBM SPSS Statistics (current documentation, version 32), the two families of post-hoc tests live in different places because they belong to different procedures. For the ANOVA case: Analyze › Compare Means › One-Way ANOVA, then the Post Hoc button opens the Multiple Comparisons dialog, where Tukey and Games-Howell both appear as tickable options in the same list. The Options button, back on the main dialog, has a "Homogeneity of variance test" checkbox that runs Levene's test, which is what should decide between the two before you even look at the pairwise output. The number that matters sits in the Multiple Comparisons table's "Sig." column, one row per pair, not the single P value in the ANOVA summary table above it; reading the ANOVA table's P value as if it already told you which groups differ is the most common misread of this output.

For the Kruskal-Wallis case, the pairwise comparisons live under a different route entirely: Analyze › Nonparametric Tests › Independent Samples (not the older Legacy Dialogs › K Independent Samples route, which returns only the omnibus H statistic with no built-in post-hoc). Running the newer procedure and then double-clicking the result to open the Model Viewer gives a Pairwise Comparisons view, with a comparison table showing each pair's standardised test statistic and an Adjusted Significance column that already has the Bonferroni correction applied across the full comparison set. The common misread here is assuming the legacy dialog's silence on post-hoc testing means SPSS "doesn't do" pairwise comparisons for Kruskal-Wallis, and falling back to a series of unadjusted Mann-Whitney tests instead.

Upload the dataset (or the relevant columns). StatsPlease's deterministic engine identifies the appropriate omnibus test and, where the result is significant, the matching post-hoc procedure from the variable type and study design, computes the result using fixed, non-LLM algorithms, and drafts the Methods/Results sentence in AMA format — the same numbers a reader would get running the tests by hand in SPSS.

References

1. McHugh ML. Multiple comparison analysis testing in ANOVA. Biochem Med (Zagreb). 2011;21(3):203-209. https://doi.org/10.11613/BM.2011.029
2. Tukey JW. Comparing individual means in the analysis of variance. Biometrics. 1949;5(2):99-114. https://doi.org/10.2307/3001913
3. Games PA, Howell JF. Pairwise multiple comparison procedures with unequal N's and/or variances: a Monte Carlo study. J Educ Stat. 1976;1(2):113-125. https://doi.org/10.3102/10769986001002113
4. Kruskal WH, Wallis WA. Use of ranks in one-criterion variance analysis. J Am Stat Assoc. 1952;47(260):583-621. https://doi.org/10.1080/01621459.1952.10483441
5. Dunn OJ. Multiple comparisons using rank sums. Technometrics. 1964;6(3):241-252. https://doi.org/10.1080/00401706.1964.10490181