StatsPlease
Validation Blog Start free
← Back to Blog

Repeated measures · ~5 min read

Friedman Test: The Non-Parametric Alternative to Repeated-Measures ANOVA

Repeated-measures ANOVA assumes the differences between timepoints are roughly normal. Real clinical markers rarely oblige.

Published

In short

When the same patients are measured at three or more timepoints, repeated-measures ANOVA is only valid if the outcome is approximately normal at each timepoint. Many clinical markers, bilirubin among them, are not.3 The Friedman test ranks each patient's own values across timepoints instead of comparing raw means, so it needs no normality assumption at all.1 Below is a real three-visit result from a Mayo Clinic trial where every timepoint failed a normality check, and what the omnibus result does and doesn't tell you once it's significant.

What Friedman trades away, and what it doesn't need

Repeated-measures ANOVA compares mean values across timepoints and assumes those means, and critically the differences between them, are approximately normally distributed. When that assumption holds, ANOVA is the more powerful choice. When it doesn't, its P value can't be trusted any more than a t-test's can on badly skewed data.

The Friedman test sidesteps the assumption entirely. Within each patient, it ranks the values at each timepoint from lowest to highest, then tests whether those ranks are distributed differently across timepoints than chance would produce.2 No normality requirement, at the cost of using rank order instead of the actual magnitude of change.

A real result: bilirubin over three visits in a Mayo Clinic trial

The Mayo Clinic's primary biliary cirrhosis (PBC) trial followed 312 patients with repeated laboratory visits.3 Among the 259 patients with a complete first three recorded bilirubin measurements, every single visit failed a Shapiro-Wilk normality check (P < .001 at each of the three visits), heavily right-skewed, as bilirubin typically is in liver disease. That rules out standard repeated-measures ANOVA before a single comparison is run.

StatsPlease output: Friedman test
VisitMedian bilirubinMean rank
Visit 11.302.008
Visit 21.201.838
Visit 31.502.154

n = 259 complete cases. Every visit failed Shapiro-Wilk (P < .001), so the engine routed to Friedman rather than repeated-measures ANOVA.

χ2(2) = 14.30 · P < .001 · Kendall's W = 0.028

A Friedman test showed a significant difference in bilirubin across the three visits (χ2[2] = 14.30, P < .001, Kendall's W = 0.028). Pairwise Wilcoxon signed-rank post-hoc (Bonferroni correction) flagged 2 of 3 pairs as significant.

Figures computed with scipy from real trial data (n = 259).

Example data: Mayo Clinic PBC trial follow-up visits (Fleming and Harrington, 1991), redistributed via the R survival package (Therneau); see references. Computed with scipy, not generated.

Kendall's W of 0.028 is small. The omnibus test is confidently significant, 259 patients gives it plenty of power, but the ranking pattern across visits is not especially consistent patient to patient. A significant P value and a small effect size are not a contradiction; they are two different questions answered honestly.

A significant Friedman test is an omnibus result, exactly like a significant ANOVA: it says at least one visit differs from the others, not which one. Here, the pairwise Wilcoxon post-hoc (Bonferroni-corrected) shows Visit 1 vs Visit 2 was not significant (P = .67), but Visit 1 vs Visit 3 (P = .015) and Visit 2 vs Visit 3 (P < .001) both were. Bilirubin rose specifically by the third visit, not steadily from the first.

Writing the AMA sentence

Report the omnibus result with its effect size, then the post-hoc detail if you ran one: "A Friedman test showed a significant difference across visits (χ2[2] = 14.30, P < .001, Kendall's W = 0.028); pairwise comparisons (Wilcoxon signed-rank, Bonferroni-corrected) indicated bilirubin at visit 3 was significantly higher than at visits 1 and 2, which did not differ from each other." Naming the specific pairs is what turns "something changed" into a result a reader can act on.

Try it yourself

Reproduce this result

The bilirubin example above was computed from a public trial follow-up dataset. Run it in either tool and the numbers match, because both are doing the same computation, not generating it.

In StatsPlease

  1. Upload the dataset as a CSV, with one bilirubin column per visit.
  2. Open the Analysis tab.
  3. Select all three visit columns as Outcome: Visit1, Visit2, Visit3. No grouping variable is used.
  4. Press the Before vs After preset button. There is no separate run step; pressing the preset screens normality at each timepoint and routes to paired t-test, Wilcoxon, or Friedman depending on what the data support.
  5. On the Results tab, the normality screen, the test actually used, and the AMA sentence, post-hoc included, are shown together.

In SPSS

  1. Check normality first: Analyze → Descriptive Statistics → Explore, add all three visit variables, tick "Normality plots with tests."
  2. Analyze → Nonparametric Tests → Legacy Dialogs → K Related Samples. Move the three visit variables into Test Variables and tick Friedman.
  3. SPSS reports the chi-square statistic and P value but does not compute Kendall's W by default; enable it under Statistics in the same dialog.
  4. Run the pairwise post-hoc separately: Analyze → Nonparametric Tests → Legacy Dialogs → 2 Related Samples (Wilcoxon) for each pair, then apply a Bonferroni correction by hand.

Frequently asked questions

When should I use the Friedman test instead of repeated-measures ANOVA?

When the same subjects are measured at three or more timepoints and the outcome is ordinal, skewed, or the sample is small. Friedman ranks each subject's own values across timepoints, so it needs no assumption about the shape of the data, unlike repeated-measures ANOVA which assumes approximate normality and equal variances of the differences.

What does Kendall's W tell you that the Friedman P value doesn't?

The P value only says whether the conditions differ. Kendall's W (0 to 1) measures how large that difference is, how consistently subjects were ranked the same way across conditions. A statistically significant Friedman result can still carry a small Kendall's W, meaning the conditions differ reliably but not by very much.

Does a significant Friedman test tell you which timepoints actually differ?

No. Like a significant ANOVA, it's an omnibus result: it says at least one timepoint differs from the others, not which pair. Identifying the specific pair requires a post-hoc test, typically pairwise Wilcoxon signed-rank tests with a Bonferroni correction to control the family-wise error rate.

You might also read

References

  1. Friedman M. The Use of Ranks to Avoid the Assumption of Normality Implicit in the Analysis of Variance. Journal of the American Statistical Association. 1937;32(200):675-701. https://doi.org/10.1080/01621459.1937.10503522
  2. Kim HY. Statistical notes for clinical researchers: nonparametric statistical methods: 2. Nonparametric methods for comparing three or more groups and repeated measures. Restorative Dentistry & Endodontics. 2014;39(4):329-332. https://doi.org/10.5395/rde.2014.39.4.329
  3. Fleming TR, Harrington DP. Counting Processes and Survival Analysis. New York: John Wiley and Sons; 1991.
  4. Therneau TM. A Package for Survival Analysis in R. R package. https://CRAN.R-project.org/package=survival

StatsPlease screens normality at every timepoint on your own uploaded data before choosing between repeated-measures ANOVA, Wilcoxon, and Friedman. Select your timepoint columns as Outcome and press the Before vs After preset; pressing it runs the analysis.

Try StatsPlease free