Test selection & reporting · ~5 min read
The Paired t-Test
The wrong test turned a null result into a "significant" one, on the same 29 patients.
Published
In short
A paired t-test and the Wilcoxon signed-rank test both compare the same patients measured twice, before and after. The choice between them depends on whether the differences between those two measurements are themselves normally distributed, not whether the raw values are. Below, the same routing question produces two different real answers in two arms of the same trial, and in one of them, using the wrong test changes whether the result is significant.
What actually needs to be normal
Both tests answer the same question: did scores change from before to after in a group of the same patients measured twice. The paired t-test checks this using each patient's individual difference score (after minus before) and requires those difference scores to be approximately normally distributed. It does not require the raw before and after values themselves to be normal, only their differences. When that difference-score assumption fails, the Wilcoxon signed-rank test, which works with the ranks of the differences rather than their raw magnitude, is the correct alternative.1
A real result, two arms, two different answers
A classic clinical dataset following young female anorexia patients recorded weight before and after one of three treatment arms: family therapy, cognitive behavioral therapy, or a no-treatment control.2 Checking the difference scores in each treated arm separately: in the family therapy arm, the differences were approximately normal (Shapiro-Wilk, P = .516). In the cognitive behavioral therapy arm, they were not (Shapiro-Wilk, P = .008).
n = 17 · mean weight change = +7.26 lb (SD 7.16) · t(16) = 4.19 · P < .001
Among patients receiving family therapy, weight increased significantly from before to after treatment (mean change, +7.26 lb; SD, 7.16), paired t(16) = 4.19, P < .001.
Figures computed with scipy from real data (family therapy arm, weight-change dataset).
The cognitive behavioral therapy arm is where the routing decision actually matters. Because its difference scores failed normality, the correctly routed test is Wilcoxon signed-rank, not the paired t-test.
Paired t-test (incorrectly routed, differences are not normal): t(28) = 2.22, P = .035
Wilcoxon signed-rank (correctly routed): W = 131.5, P = .063
Applying the paired t-test despite the failed normality check would report a significant weight change (P = .035). The correctly routed Wilcoxon signed-rank test on the same data does not reach significance (P = .063).
Figures computed with scipy from real data (cognitive behavioral therapy arm, same dataset).
Example data: the anorexia weight-change dataset of Hand DJ et al., A Handbook of Small Data Sets (Chapman & Hall, 1993), a public teaching dataset. Figures computed with scipy from real data.
That is not a rounding difference. It is a conclusion that flips across the conventional significance threshold depending on which test gets used, on the exact same 29 patients.
Writing the AMA sentence
State which test was used and name the check that justified it, not just the result. "Weight increased significantly after family therapy (P < .001)" omits the reason a paired t-test was the right choice here. The complete version states it: "Because the within-patient differences were approximately normally distributed (Shapiro-Wilk, P = .516), a paired t-test was used; weight increased significantly following family therapy (mean change, +7.26 lb; SD, 7.16), t(16) = 4.19, P < .001."
Try it yourself
Reproduce this result
In StatsPlease
- Upload the dataset as a CSV.
- Open the Analysis tab.
- Set the two paired columns as outcomes: Prewt and Postwt (no grouping variable).
- Press the Before vs After preset button. There is no separate run step; pressing the preset runs the analysis, checking normality on the difference scores and routing to the paired t-test or Wilcoxon signed-rank.
- On the Results tab, the normality check on the difference scores, the test chosen, and the AMA sentence are shown together.
In SPSS
- Compute a new variable equal to Postwt minus Prewt
- Analyze → Descriptive Statistics → Explore, request Shapiro-Wilk on that difference variable specifically
- Analyze → Compare Means → Paired-Samples T Test if differences are normal, or Analyze → Nonparametric Tests → Related Samples (Wilcoxon) if they are not
- Report whichever test the difference-score check actually supported
Frequently asked questions
Does a paired t-test require the raw before and after scores to be normally distributed?
No. It requires the difference between each patient's before and after score to be approximately normally distributed. The raw values themselves can be non-normal as long as the differences are not.
What test should replace a paired t-test when difference scores aren't normal?
The Wilcoxon signed-rank test, which uses the ranks of the differences rather than their raw magnitude, and does not require the differences to follow a normal distribution.
Can using a paired t-test instead of Wilcoxon change whether a result is significant?
Yes. On the same 29-patient dataset used in this article, an incorrectly applied paired t-test returned P = .035 while the correctly routed Wilcoxon signed-rank test on identical data returned P = .063, crossing the conventional significance threshold in opposite directions.
You might also read
References
- Nahm FS. Nonparametric statistical tests for the continuous data: the basic concept and the practical use. Korean Journal of Anesthesiology. 2016;69(1):8-14. https://doi.org/10.4097/kjae.2016.69.1.8
- Hand DJ, Daly F, McConway K, Lunn D, Ostrowski E, eds. A Handbook of Small Data Sets. London: Chapman & Hall; 1993. Dataset 285.
StatsPlease checks the normality of your difference scores specifically before choosing between a paired t-test and Wilcoxon signed-rank.
Try StatsPlease free