Sample Size and Power: How to Justify Your n Before You Collect Data

"Was this study large enough?" is one of the first questions a reviewer asks, and one of the most common reasons a manuscript is sent back. A study that is too small can miss a real, clinically important effect and report a falsely reassuring negative. It can also, less obviously, make a significant result less trustworthy: when power is low, the significant findings that do emerge tend to overestimate the true effect.³ A power analysis is how you avoid both traps, and it belongs in the planning stage, before any data exist.

What a power analysis actually answers

Statistical power is the probability that your study will detect an effect of a given size, if that effect is really there. A study with 80% power has an 80% chance of returning a statistically significant result when the effect it was designed to find is genuinely present, which also means a 20% chance of missing it (a type II error).⁵ An a-priori power analysis inverts that relationship: you fix the power you want and solve for the sample size that delivers it.

The ethical case is as strong as the statistical one. Enrol too few patients and you risk a study that cannot answer its own question, exposing participants to research burden for a result nobody can trust. Enrol far more than you need and you expose extra patients to risk for no added knowledge. The right sample size is the smallest one that can properly evaluate the study aim.⁵

The four numbers you need

Every a-priori calculation comes down to four decisions. Settle them in order and the sample size follows.

1. The effect size you care about. This is the one that trips people up. It should be the smallest effect that would be clinically meaningful, not the large effect you are hoping for or the effect a pilot happened to show. Powering for an optimistic effect produces a small, underpowered study that will miss a smaller but still important difference.⁵ Effect size can be entered as a standardized value (Cohen's d, Pearson r) or built from clinical terms: a mean difference and its standard deviation, or two proportions. Cohen's conventions are a rough guide when you have nothing else: d of 0.2 is small, 0.5 medium, 0.8 large.¹

2. The significance level (alpha). The probability of a false positive. Convention is a two-sided .05. Lower it only if you have a specific reason, such as multiple comparisons.

3. The power you want. Conventionally at least 80%, and 90% is better when feasible.⁵ Higher power costs more patients, so it is a trade-off between rigour and recruitment.

4. The test and design. A two-sample t-test, a paired comparison, a one-way ANOVA, a correlation, and a comparison of two proportions each need a different formula. The sample size for a paired design is usually much smaller than for the equivalent independent-groups design, so the design choice matters before the arithmetic.

Consider a two-arm surgical trial comparing a continuous outcome between groups. You decide the smallest difference worth detecting corresponds to a medium standardized effect, Cohen's d of 0.5, and you want 80% power at a two-sided .05. StatsPlease solves it deterministically:

Sample-size and power figures computed with scipy/statsmodels (the same solvers behind G*Power and R's pwr package); no dataset is required for a planning calculation.

The reviewer trap: post-hoc power

Here is the single most common power mistake in the clinical literature, and it is not underpowering. It is computing power after the study, from the effect the study happened to observe, to explain a non-significant result: "the study had only 40% power to detect the observed difference." This post-hoc or observed power is uninformative, and often circular.² Observed power is a direct mathematical function of the p-value: a non-significant result always corresponds to low observed power, so the statement adds nothing you did not already know from the p-value itself.⁴

The correct tool for interpreting a completed study is the confidence interval. If a trial finds no significant difference, the confidence interval on the effect tells you what the data can and cannot rule out: a tight interval around zero is genuine evidence of no important effect, while a wide interval means the study was simply inconclusive.² Power belongs before the experiment; confidence intervals belong after it. Do not cross the two.⁴

How to report it in AMA format

State all four inputs and the resulting sample size in one or two sentences in your methods. A complete example: "A sample size of 64 patients per group (128 total) was required to detect a between-group difference of Cohen's d = 0.5 with 80% power at a two-sided significance level of .05. The effect size was chosen as the smallest difference judged clinically meaningful." If you inflated the target to allow for dropout, say so: "Allowing for 15% attrition, we planned to enrol 75 per group."

Keep the leading zero off the significance level (.05, not 0.05) and on effect sizes and power (0.5, 0.80). Italicise the statistic symbols (d, r, n). Naming the effect size as the minimal clinically important difference, rather than an expected or pilot-derived value, is exactly what a statistical reviewer looks for.⁵

References

1. Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates; 1988.
2. Heckman MG, Davis JM 3rd, Crowson CS. Post Hoc Power Calculations: An Inappropriate Method for Interpreting the Findings of a Research Study. The Journal of Rheumatology. 2022;49(8):867–870. https://doi.org/10.3899/jrheum.211115
3. Button KS, Ioannidis JPA, Mokrysz C, et al. Power failure: why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience. 2013;14(5):365–376. https://doi.org/10.1038/nrn3475
4. Hoenig JM, Heisey DM. The Abuse of Power: The Pervasive Fallacy of Power Calculations for Data Analysis. The American Statistician. 2001;55(1):19–24. https://doi.org/10.1198/000313001300339897
5. Schober P, Vetter TR. Sample Size and Power in Clinical Research. Anesthesia & Analgesia. 2019;129(2):323. https://doi.org/10.1213/ANE.0000000000004375