StatsPlease
Validation Blog Start free
← Back to Blog

Test selection & reporting · ~5 min read

The Independent t-Test

A null result still deserves a full report. Here's what that looks like done right.

Published

In short

An independent (two-sample) t-test compares means between two unrelated groups, and it is only the right test once normality has actually been checked in each group, not assumed. Below is a full worked example on real, public clinical trial data: patient age compared between those with and without spider angiomata in the Mayo Clinic PBC dataset. Both groups clear Shapiro-Wilk, so the comparison is correctly routed to a t-test rather than Mann-Whitney U, and the honest result here is a non-significant one, reported the same rigorous way a significant result would be.

When a t-test is the right test, and when it isn't

An independent t-test asks whether two unrelated groups have different means on a continuous outcome. It has two assumptions worth actually checking rather than asserting: each group's data should be reasonably normally distributed, and the two groups' variances should be reasonably similar. When normality fails in either group, the comparison routes to Mann-Whitney U instead.2 A different post covers that test and how the two relate.

The variance assumption is a separate check, and it changes which version of the t-test is appropriate: Student's t-test assumes equal variances; Welch's t-test does not. Using a preliminary test of variances to decide between them, however, is itself discouraged in the methodological literature: choosing Welch's by default performs about as well as Student's when variances are genuinely equal, and meaningfully better when they are not.3 That is the convention used below.

A computed example: age by spider angiomata in PBC

Spider angiomata are a visible sign of chronic liver disease. Using the Mayo Clinic's primary biliary cirrhosis (PBC) trial data, 312 randomised patients with complete baseline records,1 age was compared between the 90 patients with spider angiomata at baseline and the 222 without. Shapiro-Wilk did not reject normality in either group (P = .057 without, P = .568 with), so a t-test is the correct choice here, not an assumed one.

StatsPlease output: independent t-test
GroupnMean ageSD
Spider angiomata present9049.2 years10.5
Spider angiomata absent22250.4 years10.6

Levene's P = .846 (variances not unequal) · Welch's t(165.9) = -0.89 · P = .38 · Cohen's d = 0.11 · 95% CI of mean difference, -3.8 to 1.4

Mean age did not differ significantly between patients with spider angiomata (49.2 [SD, 10.5] years) and those without (50.4 [SD, 10.6] years), Welch's t(165.9) = -0.89, P = .38, Cohen's d = 0.11 (95% CI of the mean difference, -3.8 to 1.4).

Figures computed with scipy from real data (PBC trial, Mayo Clinic).

Example data: Mayo Clinic primary biliary cirrhosis (PBC) trial (Fleming and Harrington, 1991); see references. Computed with scipy, not generated.

This is not the cherry-picked case where everything comes out significant. It is the ordinary one: two groups that turn out not to differ much on this particular baseline variable, reported with the same effect size and confidence interval a significant result would get. A P value of .38 is not evidence the groups are the same, only that this sample did not detect a difference, and reporting Cohen's d and the CI alongside it is what lets a reader judge how large a true difference this sample could plausibly have missed.3

Writing the AMA sentence

The reporting pattern holds regardless of the result: state both group means with SDs, name which t-test was used and why, give the test statistic with degrees of freedom, the P value, and an effect size with its confidence interval. "Mean age was similar between groups (P = .38)" is not a complete sentence by AMA convention; the version above is.

Try it yourself

Reproduce this result

The example above was computed from a public dataset. Run it in either tool and the numbers match, because both are doing the same computation, not generating it.

In StatsPlease

  1. Upload the dataset as a CSV.
  2. Open the Analysis tab.
  3. Set age as the Outcome and spiders as the Group.
  4. Press the Group Comparison preset button. There is no separate run step; pressing the preset runs the analysis, screening normality and variance, then routing to the t-test or Mann-Whitney.
  5. On the Results tab, the assumption checks, the test chosen, and the AMA sentence with the effect size and CI are shown together.

In SPSS

  1. Analyze → Descriptive Statistics → Explore, split by group, check Shapiro-Wilk.
  2. Analyze → Compare Means → Independent-Samples T Test.
  3. Read Levene's test in the output table first.
  4. Take the Equal/Not equal variances row accordingly.
  5. Compute Cohen's d and its CI by hand; SPSS does not report it by default.

Frequently asked questions

Does an independent t-test require the raw data to be perfectly normally distributed?

It requires each group to be reasonably close to normally distributed, checked with a test such as Shapiro-Wilk. When normality fails, Mann-Whitney U is the correctly routed alternative instead.

Should you always use Student's t-test or Welch's t-test by default?

Welch's t-test is the recommended default. It performs comparably to Student's t-test when variances are genuinely equal and performs better when they are not, so choosing it by default avoids relying on a preliminary variance test to decide.

Is a non-significant t-test result still worth reporting in full?

Yes. A non-significant P value should still be reported with the effect size and confidence interval, since these tell a reader how large a true difference the study could plausibly have missed, which a P value alone does not.

You might also read

References

  1. Fleming TR, Harrington DP. Counting Processes and Survival Analysis. New York: John Wiley and Sons; 1991.
  2. Nahm FS. Nonparametric statistical tests for the continuous data: the basic concept and the practical use. Korean Journal of Anesthesiology. 2016;69(1):8-14. https://doi.org/10.4097/kjae.2016.69.1.8
  3. Delacre M, Lakens D, Leys C. Why psychologists should by default use Welch's t-test instead of Student's t-test. International Review of Social Psychology. 2017;30(1):92-101. https://doi.org/10.5334/irsp.82
  4. Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates; 1988.

StatsPlease checks normality and variance homogeneity on your own uploaded groups, routes to the correct test, and drafts the AMA sentence with the effect size and CI already computed, not left for you to add by hand. Set your outcome and grouping variable and press the Group Comparison preset; pressing it runs the analysis.

Try StatsPlease free