StatsPlease
Validation Blog Start free
← Back to Blog

Interpretation · ~7 min read

How to Show a "Not Significant" Result Actually Ruled Out a Clinically Important Difference

Statistics for clinical researchers and surgical trainees

Published

In short

A confidence interval that fails to reach significance is not automatically uninformative. If the entire interval sits inside the range of "not clinically important," that is a genuinely strong, positive finding, not a shrug. A 2025 method formalises this check: compare the interval's bounds directly to the MCID and, when the interval straddles it, calculate exactly what proportion still supports a meaningful effect. Below is the calculation worked through on a real trial result, followed by the harder straddling case done by hand.

The formula, in full

Call the confidence interval's lower and upper bounds δL and δU, and the MCID δ.2 There are exactly three possible relationships between them:

  1. δL > δ: the whole interval clears the threshold. Clinically important, not just statistically detectable.
  2. δU < δ: the whole interval sits below the threshold. Not clinically important, whatever the P value says.
  3. δL < δ < δU: the interval straddles it. Calculate Pδ = (δU minus δ) / (δU minus δL), the proportion of the interval above the threshold.

Case 2 is the one most people skip past, because a non-significant P value already feels like the end of the analysis. It should not be. A confidence interval that positively excludes the MCID (on both sides of zero if the MCID could plausibly favour either group) is a real, reportable finding: the data are precise enough to rule out a meaningful difference, not just too imprecise to find one.1

Case 2, on a real trial

The UKUFF trial compared open and arthroscopic rotator cuff repair in patients over 50 with a symptomatic degenerative tear, with the Oxford Shoulder Score at 24 months as the primary outcome.3 The trial pre-specified 3 points as the clinically important difference on that scale. The result: a mean difference of -0.76 points, 95% CI, -2.75 to 1.22, P = .452.3

Working through Case 2

δ = 3 (either direction). δL = -2.75, δU = 1.22. Since δU = 1.22 < 3, and δL = -2.75 > -3, the entire interval sits inside the plus-or-minus 3 point zone.

QuantityValueAgainst δ = 3
Mean difference-0.76 pointsInside
Lower bound δL-2.75Above -3
Upper bound δU1.22Below +3

The full interval rules out a clinically important difference favouring either procedure, not merely "no significant difference," but a positive exclusion of the threshold on both sides.

Real trial result (Carr et al., 2015); the trial's own authors reported this same conclusion directly.

Values quoted from Carr AJ, Cooper CD, Campbell MK, et al., the UK Rotator Cuff Surgery (UKUFF) randomised trial, Health Technology Assessment. 2015;19(80). Read from the published paper, not computed from a StatsPlease run.

That is a materially stronger statement than "no significant difference was found," and it is available directly from numbers already in the paper. No new data, no new test, just checking the two ends of a confidence interval against a threshold that was, in this case, decided in advance.

Case 3: when the interval straddles the threshold

Most results are not this tidy. Suppose a trial on a different shoulder outcome, the Oxford Shoulder Score after arthroplasty rather than repair, reported a between-group mean difference of 8 points, 95% CI, 2–14. The MCID for that specific population and instrument has been established at 6.9 points.4

  • δL = 2, δU = 14, δ = 6.9
  • Is δL > δ? 2 > 6.9 is false.
  • Is δU < δ? 14 < 6.9 is false.
  • So Case 3 applies. Pδ = (14 minus 6.9) / (14 minus 2) = 7.1 / 12 = 0.592

(Illustrative CI, constructed to demonstrate the calculation; the MCID value, 6.9, is real and cited, the trial result is not.)

About 59% of the plausible range of effect sizes clears the threshold, and about 41% does not. That is a genuinely different, more honest statement than either "significant" (which says nothing about whether the effect matters) or "the point estimate exceeds the MCID" (which ignores that a third of the interval's lower half does not). Report the number, and say plainly what it does and does not establish: it describes how compatible the data are with a clinically important effect, not a probability that the true effect is clinically important.

Note what the arithmetic is and is not. It is a ratio of two subtractions, reproducible on paper in ten seconds, and it will come out identically in R, in SPSS, or on the back of a clinic list. Ask ChatGPT to do the same job on your interval and you get a plausible-looking number produced by a language model rather than by arithmetic, which is exactly the failure mode this whole check is designed to avoid. Compute it, do not generate it.

What to write in your results section

State which case applies, not just the raw numbers. For Case 2, say so explicitly: "the 95% CI excluded the pre-specified MCID of δ," which is a stronger and more specific claim than "not statistically significant." For Case 3, report Pδ alongside the interval and name the method: "δ = [value], based on [source]; Pδ = [value], indicating [proportion] of the plausible effect range exceeds this threshold," citing the calculation.2 Either way, the MCID itself needs its own citation and needs to come from a population reasonably similar to the one in front of you, not borrowed from wherever it is easiest to find one.

Common questions

Can a "not significant" result still meaningfully rule out a clinically important difference?

Yes. If a confidence interval sits entirely within the range considered not clinically important, that is a positive, reportable finding: the data were precise enough to rule out a meaningful difference, not merely too imprecise to detect one.

What is Pδ?

Pδ is the proportion of a confidence interval that falls beyond the minimal clinically important difference (MCID) threshold, calculated directly from the interval's lower and upper bounds relative to that threshold.

When does a confidence interval need to be checked against the MCID using the full range rather than just the point estimate?

Whenever the interval straddles the MCID threshold rather than sitting entirely above or below it. In that case, the point estimate alone does not convey how much of the plausible effect range actually supports a clinically important result.

Try it yourself

Get the interval you need: in SPSS or StatsPlease

Both checks above start from one thing: a confidence interval around a between-group difference. Here is how to produce that interval from your own data in either tool, so you have δL and δU to compare against your cited MCID.

In SPSS

  1. Open your dataset with one continuous outcome column (the score) and one two-level grouping column (the procedure).
  2. Check normality first: Analyze → Descriptive Statistics → Explore. Put the score in the Dependent List and the procedure in the Factor List, tick "Normality plots with tests," click OK.
  3. Run the comparison: Analyze → Compare Means → Independent-Samples T Test. Move the score into Test Variable(s) and the procedure into Grouping Variable, then Define Groups and click OK.
  4. Read the "Mean Difference" and the "95% Confidence Interval of the Difference" Lower and Upper columns. Those two numbers are δL and δU. Compare them to your cited MCID by hand, and compute Pδ yourself if the interval straddles it. SPSS does not report an effect size here, so calculate Cohen's d separately.

In StatsPlease

  1. Upload the CSV.
  2. Open the Analysis tab.
  3. Set the score as the Outcome and the procedure as the Group.
  4. Press the Group Comparison preset button. There is no separate run step: pressing the preset is what runs the analysis. You pick the preset, and the engine picks the specific test (it screens normality with Shapiro-Wilk plus Levene's, then routes).
  5. Read the Results tab. The mean difference, its 95% CI, the exact P value, and the effect size come back together, so δL and δU are in front of you without a second lookup. Output is AMA by default, or APA via the Reporting Style picker.

Compare: both tools return the identical interval, because both run the same test on the same numbers. The difference is assembly: SPSS makes you hunt the Lower and Upper columns and compute the effect size separately, while StatsPlease returns the difference, the interval, the P value, and the effect size in one formatted sentence, computed from your data, not generated. The MCID comparison is still yours to make, and the MCID still needs its own citation.

You might also read

References

  1. Jaeschke R, Singer J, Guyatt GH. Measurement of health status. Ascertaining the minimal clinically important difference. Controlled Clinical Trials. 1989;10(4):407–415. https://doi.org/10.1016/0197-2456(89)90005-6
  2. De Cassai A. The ratio of confidence interval above the minimal clinically important difference. Anaesthesia. Published online October 10, 2025. https://doi.org/10.1111/anae.70028
  3. Carr AJ, Cooper CD, Campbell MK, et al. Clinical effectiveness and cost-effectiveness of open and arthroscopic rotator cuff repair [the UK Rotator Cuff Surgery (UKUFF) randomised trial]. Health Technology Assessment. 2015;19(80):1–218. https://doi.org/10.3310/hta19800
  4. Liu P, Afzal I, Asopa V, Clement ND, Patel V. Changes and thresholds in the Oxford Shoulder Score following shoulder arthroplasty: minimal clinically important difference, minimal important and detectable changes, and patient-acceptable symptom state. Shoulder & Elbow. 2024. https://doi.org/10.1177/17585732231176423

Start this check from numbers you already have.

StatsPlease reports the full confidence interval alongside every effect size it computes from your own data, so the MCID comparison starts from numbers already in front of you rather than a calculation to do separately by hand. Upload your CSV, set your Outcome and Group, and press the Group Comparison preset: pressing it runs the analysis and returns the difference, its 95% CI, the exact P value, and the effect size, computed from your data, not generated.

Try StatsPlease free