Sample Size in Clinical Research: What It Means for Your Statistical Tests

Sample size is not just a count of how many patients you recruited. It is the single thing that most directly decides which statistical tests are valid, how likely you are to detect a real difference, and what conclusions you can responsibly draw. Most surgical studies, audits, and trainee projects work with fewer than 50 patients in total — often far fewer. Understanding what that means is not optional.

Why sample size changes everything

When groups are large — roughly 100 or more each — a rule called the central limit theorem means the averages behave in a predictable, bell-shaped way even if the raw data do not. Parametric tests — tests that assume the data follow a normal, bell-shaped curve — work well at that size. When groups are small — under about 30 each — that protection disappears: averages become unreliable summaries, you can no longer assume a normal shape, and parametric tests can give misleading results. Most clinical research sits in this small-sample range.

The "fewer than 30" rule

When you have fewer than about 30 people per group, use a non-parametric test unless a normality test confirms the data are normal. Run a Shapiro-Wilk test on each continuous outcome.¹ If its p-value is above 0.05, a parametric test is acceptable; if it is 0.05 or below, use the non-parametric version. This is not a weakness in your study — it is simply a constraint that decides your analysis, and stating it openly in your methods makes your paper stronger, not weaker.

Test selection by sample size

Statistical power when groups are small

Power is the chance of detecting a real difference when one truly exists. The usual target is 80%. Consider a study of patients with severely reduced cardiac function (ejection fraction ≤ 20%). Only 23 patients met this criterion. This is a common situation in clinical research: the subgroup of interest is small, and that small size limits what you can detect. When groups are small, power drops sharply for moderate differences. For a medium difference, roughly 10 people per group gives only about 18% power, 30 per group about 47%, and you need about 64 per group to reach the 80% target. The general point — that the sample size needed depends heavily on the effect size and the power you want — is shown clearly by Zhu in a detailed comparison of methods for this test,² and the small/medium/large effect categories come from Cohen.³ The practical message is not that small studies are worthless. It is that they can only reliably detect large effects, so describe them as pilot or exploratory and save firm claims for properly powered work.

Small-sample results are summarised with the median and the interquartile range (the IQR, the middle 50% of the values), and StatsPlease flags the limited power, as in the output below:

Example data: Vanderbilt University Department of Biostatistics public teaching datasets (hbiostat.org/data). Figures computed with scipy from real data.

What reviewers expect

For any study with fewer than about 50 patients in total, reviewers will ask: did you justify your sample size, either with a power calculation or by clearly calling the study exploratory? Did you test for normality and report the result? Did you choose parametric or non-parametric tests appropriately for your size? Did you report effect sizes rather than just p-values? And are your conclusions suitably cautious? Preparing for these questions in your methods section prevents most statistics-related revisions.

Template language for small studies

You can adapt this: "Given the exploratory nature of this study and a total sample size of n = [X], non-parametric tests were used throughout. Normality was assessed with the Shapiro-Wilk test. Results should be interpreted with caution pending confirmation in a larger cohort."

How StatsPlease handles small samples

StatsPlease detects how many people are in each group before choosing a test, runs the Shapiro-Wilk check automatically, and defaults to a non-parametric test when groups are small or the data are not normal. It flags comparisons that are likely underpowered in the output and includes the appropriate cautious wording in the generated methods statement.

References

1. Mishra P, Pandey CM, Singh U, Gupta A, Sahu C, Keshri A. Descriptive Statistics and Normality Tests for Statistical Data. Annals of Cardiac Anaesthesia. 2019;22(1):67–72. https://doi.org/10.4103/aca.ACA_157_18
2. Zhu X. Sample size calculation for Mann-Whitney U test with five methods. International Journal of Clinical Trials. 2021;8(3):184–195. https://doi.org/10.18203/2349-3259.ijct20212840
3. Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates; 1988.