Which Statistical Test Should I Use?

"Which test should I use?" sounds like it should have a discretionary answer, the way "which journal should I submit to?" does. It does not. Once you can state your research question precisely and you know what your data look like, the test is close to fully determined: the structure of the comparison narrows the field to one or two candidates, and an assumption check settles which of those is valid.¹ Choosing differently is not a stylistic decision; it is the kind of error a statistical reviewer is specifically reading for.

The six questions below cover the large majority of analyses in clinical papers. Each entry names the question, the test, and the single check that decides the routing, then links to a full guide with a worked example computed from real, public data.

Find your research question

1. Comparing a continuous outcome between two independent groups

Blood loss in open versus laparoscopic cases, length of stay with and without a complication: one continuous outcome, two unrelated groups of patients. Normality is the assumption that decides the routing. When the outcome is roughly normal in each group, the independent t-test compares the means. When either group is small or visibly skewed, or the outcome is ordinal, Mann-Whitney U compares the groups by rank instead, and it is the safer default when in doubt.²

2. Comparing a continuous outcome across three or more groups

Three anaesthetic protocols, four disease stages: the same comparison extended past two groups. The test is one-way ANOVA, which again rests on approximate normality within each group. Its result is an omnibus: a significant ANOVA says that at least one group differs, not which one, so it must be followed by a post-hoc test that names the differing pairs while keeping the overall error rate controlled. Running separate t-tests between every pair instead inflates the false-positive rate, and it is one of the first things a reviewer checks.

3. The same subjects measured twice

Pain scores before and after an intervention, a biomarker at baseline and at follow-up: both measurements come from the same patients, so the groups are not independent and the two-group tests above do not apply. The deciding check moves as well. It is the normality of the difference scores, one per patient, not of either measurement on its own. When the differences are roughly normal, use the paired t-test; when they are not, the Wilcoxon signed-rank test is its rank-based counterpart.

4. The relationship between two continuous variables

Does operative time track with BMI? Does one lab value rise with another? That is correlation, and the routing again turns on normality, this time of both variables. Pearson's r is appropriate when both are approximately normal and the relationship looks linear on a scatter plot; Spearman's rank correlation is the choice otherwise, including whenever either variable is ordinal or the plot shows influential outliers.

5. Association between two categorical variables

Complication (yes or no) by surgical technique, smoking status by disease stage: two categorical variables cross-tabulated into a contingency table. The test is chi-square, or Fisher's exact when expected cell counts are small. The expected counts are the check that decides it: the chi-square approximation is only trustworthy when every cell's expected count is adequate (the usual working rule is at least 5), and below that Fisher's exact computes the probability directly rather than approximating it.

6. Predicting a yes/no outcome from several variables at once

When the question stops being one comparison and becomes several influences at once (does the complication depend on age, diabetes, and operative time together?), the tool is multivariable logistic regression. It returns an adjusted odds ratio for each predictor, and "adjusted" has a precise meaning: each odds ratio is estimated with the other variables in the model held constant. That is what separates it from the crude, one-variable-at-a-time associations above, and it is why the adjusted and unadjusted numbers for the same predictor can differ substantially.

Writing it up

Whichever branch your data land in, the write-up follows one rule: name the test and name the check that justified it. "The Mann-Whitney U test was used because Shapiro-Wilk indicated non-normal distribution of blood loss" closes a methods query before it is raised; "a t-test was performed" invites one. The same holds in the other direction: a t-test earned by data that actually passed its normality check is worth saying so, in one sentence. And whatever the test, report the statistic with its exact P value and an effect size, whether or not the result is significant.

Frequently asked questions

How do I know if my data is normal?

Look at a histogram first: a roughly symmetric, bell-shaped distribution supports a parametric test, while a long tail to one side does not. Then run a Shapiro-Wilk test, which returns a P value for the hypothesis of normality. Treat the two together rather than reading the P value alone: in very small samples Shapiro-Wilk lacks the power to detect real skew, and in very large samples it flags trivial deviations a t-test tolerates. StatsPlease runs Shapiro-Wilk on every group automatically before it selects a test.

What happens if my data isn't normally distributed?

You use the rank-based counterpart: Mann-Whitney U instead of the independent t-test, the Wilcoxon signed-rank test instead of the paired t-test, Spearman instead of Pearson.² These tests make no assumption about the shape of the distribution, and using one where a parametric test would also have been valid is conservative, not wrong. The result is reported the same way, with the test statistic, the exact P value, and an effect size.

Do I have to choose the test myself?

Not in StatsPlease. You choose the preset that matches your design (Group Comparison, Before vs After, Correlation, Category Patterns, or Regression) and the engine screens the assumptions and routes to the correct test, following the same logic this page describes. That is different from asking ChatGPT or another LLM, which will name a test and produce plausible-looking numbers with no fixed computation behind them. In StatsPlease every statistic is computed by scipy and statsmodels; the AI never computes a number.

Is a non-significant result still worth reporting?

Yes. A non-significant P value means this sample did not detect a difference, not that no difference exists, and it deserves the same complete report: the test used, the check that justified it, the exact P value, and the effect size with its confidence interval. That is what lets a reader judge how large a true effect the study could plausibly have missed. The independent t-test guide walks through a real non-significant result reported in full.

References

1. du Prel JB, Röhrig B, Hommel G, Blettner M. Choosing statistical tests: part 12 of a series on evaluation of scientific publications. Deutsches Ärzteblatt International. 2010;107(19):343-348. https://doi.org/10.3238/arztebl.2010.0343
2. Nahm FS. Nonparametric statistical tests for the continuous data: the basic concept and the practical use. Korean Journal of Anesthesiology. 2016;69(1):8-14. https://doi.org/10.4097/kjae.2016.69.1.8