Correlation

What Pearson actually assumes

Pearson's r measures the strength of a straight-line relationship between two continuous variables, and it does this by working directly with the raw values. That makes it sensitive to exactly the kind of skew and outliers common in clinical lab values, where most patients cluster in a normal range and a smaller group of severely affected patients stretches the distribution's tail out considerably. Spearman's rho sidesteps this by first converting both variables to ranks and correlating the ranks instead of the raw numbers, a transformation that makes it far less sensitive to a handful of extreme values.¹

A real result: glucose and long-term glycemic control

Vanderbilt's diabetes dataset pairs a same-day stabilized glucose reading (stab.glu) with glycosylated hemoglobin (glyhb), a marker of average blood glucose over roughly the preceding three months, in 403 patients screened for diabetes risk.² Checking normality first: Shapiro-Wilk rejects it decisively for both variables (stab.glu, W = 0.65, P < .001; glyhb, W = 0.72, P < .001), driven by the same pattern in both, a cluster of patients with normal values and a smaller group of diabetic patients with values far out in the tail.

Example data: Vanderbilt University Department of Biostatistics public teaching datasets (hbiostat.org/data). Figures computed with scipy from real data.

That 0.75-versus-0.53 gap is not noise. Pearson's coefficient is being pulled upward by the same handful of severely hyperglycemic patients that broke the normality check in the first place. Running Pearson here by default, because it is the option most software opens with, would report a relationship measurably stronger than the rank-based evidence actually supports.

Writing the AMA sentence

State which coefficient was used and why in the same sentence as the result. "Stabilized glucose and glycosylated hemoglobin were correlated (r = 0.75)" omits the one piece of information a reviewer needs to trust the number: whether the distributional assumption behind it actually held. The version that survives review names the check: "Because both variables departed from normality (Shapiro-Wilk, P < .001), Spearman's rank correlation was used; stabilized glucose and glycosylated hemoglobin were significantly correlated (ρ = 0.53; 95% CI, 0.45–0.60; P < .001)."

Frequently asked questions

How different can Pearson's r and Spearman's rho be on the same data?

Substantially. On real skewed clinical data, the two coefficients can diverge by 0.2 points or more, because Pearson works with raw values and is sensitive to extreme values, while Spearman works with ranks and is much less affected by them.

How do you decide between Pearson and Spearman correlation?

Check normality in both variables, typically with Shapiro-Wilk. If both are reasonably normal, Pearson is appropriate. If either departs significantly from normality, Spearman's rank correlation is the correctly routed alternative.

Does most statistical software choose the right correlation coefficient automatically?

No. Most software defaults to Pearson unless specifically told otherwise, regardless of whether the data's distribution actually supports it.

References

1. Nahm FS. Nonparametric statistical tests for the continuous data: the basic concept and the practical use. Korean Journal of Anesthesiology. 2016;69(1):8-14. https://doi.org/10.4097/kjae.2016.69.1.8
2. Data obtained from http://hbiostat.org/data courtesy of the Vanderbilt University Department of Biostatistics.
3. Bonett DG, Wright TA. Sample size requirements for estimating Pearson, Kendall, and Spearman correlations. Psychometrika. 2000;65(1):23-28. https://doi.org/10.1007/BF02294183