McNemar's Test: Paired Categorical Data

The distinction nobody skips for a mean, and almost everyone skips for a proportion

Ask a resident whether operative time before and after adopting a new approach needs a paired or an independent t-test, and they'll get it right without thinking hard: same patients, two measurements, paired test. Nobody reaches for the independent-samples t-test on that data by accident. But swap the outcome from a continuous measurement to a binary one, complication yes or no, symptom present or absent, and the same design gets run through an ordinary chi-square test constantly. The chi-square test of independence assumes every observation in the table comes from a different, unrelated unit. Repeat the same 80 patients across two columns and that assumption is false by construction, but nothing in the software stops you from ticking the box anyway.

The paired/unpaired instinct that's automatic for continuous outcomes doesn't seem to transfer to proportions, and it isn't a rare slip. A short 1976 paper titled, plainly, "The incorrect use of chi-square analysis for paired data," exists because the error was common enough in the immunology literature of the time to warrant a dedicated note pointing it out.² The underlying mechanics haven't changed since.

What McNemar's test actually evaluates

Lay the two paired measurements out as a 2×2 table and the four cells split into two kinds. The diagonal cells are patients whose status was the same both times, positive-positive or negative-negative. The off-diagonal cells are the ones who changed: positive-then-negative, or negative-then-positive. McNemar's test, introduced by Quinn McNemar in 1947, ignores the diagonal entirely and asks a single question of the two off-diagonal cells: among the patients who changed, did they change in both directions about equally, or did the change run mostly one way?¹ If a treatment has no real effect on the outcome, patients should drift from positive to negative and from negative to positive in roughly equal numbers, just from noise. A lopsided split between those two discordant cells is the actual signal McNemar's test is built to detect.

That's also why it discards information a chi-square test would otherwise use. An ordinary chi-square test of independence works from the row and column marginal totals, treating the "before" column and the "after" column as if they were two different samples of patients. Doing that on paired data doesn't just violate the independence assumption, it also throws away the pairing itself: the same patient's before-and-after link is the whole point of a matched design, and a marginal-totals comparison can't see it.

A worked example: donor-site numbness that resolves, mostly

The following figures are illustrative, not from a real dataset. A hand surgery unit follows 84 patients after fibula free flap harvest and records, at 3 months and again at 12 months, whether each patient still reports donor-site numbness (yes/no).

The diagonal cells, 22 patients numb at both visits and 28 numb at neither, don't enter the calculation. What matters is the 30 patients who went from numb to not-numb, against the 4 who went the other way. McNemar's chi-square statistic is (30−4)² ÷ (30+4) = 19.88 on 1 degree of freedom, a result reported in the manuscript as a highly significant reduction in numbness between 3 and 12 months, not as "84 patients at 3 months vs 84 patients at 12 months differed."

Now run the wrong test on the same numbers. Treat the 3-month column and the 12-month column as two independent groups of 84, cross-tabulate time point against numbness status using only the marginal totals (52 vs 32 numb-status split at 3 months, 26 vs 58 at 12 months), and an ordinary chi-square test of independence returns 16.18 on 1 degree of freedom. That's a different number answering a different, incorrect question: it has quietly doubled the effective sample size to 168 by counting each of the 84 patients twice, as if the 3-month numbness status and the 12-month numbness status came from different people. In this example both calculations happen to land on a significant result, but that's not guaranteed, and the naive version's p-value doesn't carry the properties it's assumed to carry, because the independence assumption it depends on doesn't hold for this design.²

When the discordant pairs are few

McNemar's chi-square statistic is only an approximation, and it needs enough discordant pairs to be trustworthy. A commonly cited rule of thumb is that once b + c, the two discordant cells combined, drops below about 25, the chi-square approximation stops being reliable, and the standard advice has long been to switch to the exact binomial version of the test instead. A 2013 analysis evaluating 9,595 parameter combinations by exhaustive enumeration, not random simulation, complicated that advice: the exact conditional test turned out to be needlessly conservative, holding its Type I error rate well below the nominal 5% even at moderate sample sizes and losing power as a result. The same study recommended the mid-p version of the test, or the plain asymptotic chi-square without a continuity correction, as the better-performing options for applied use — it found the continuity-corrected and exact conditional tests both needlessly conservative, with poor power, and recommended against reporting either.³ The practical takeaway for a manuscript with a thin discordant-pairs count isn't "always use the exact test," it's to know which version your software actually ran and to name it, because "exact," "mid-p," and "chi-square with continuity correction" are three different calculations that can disagree at small numbers.

One more limit worth knowing: McNemar's test itself only handles a 2×2 table, two paired categories. A paired outcome with three or more categories, three raters, or an ordinal scale measured twice, needs the McNemar-Bowker test of symmetry, McNemar's generalisation to larger square tables, not McNemar's test itself.⁴

How to run it — SPSS vs StatsPlease

In current IBM SPSS Statistics (documentation verified against version 32), McNemar's test lives inside Crosstabs, not as its own separate procedure: Analyze ▸ Descriptive Statistics ▸ Crosstabs. Put one paired measurement (the "before" or first-rater variable) into the Row(s) box and the other (the "after" or second-rater variable) into the Column(s) box, then open Statistics and tick McNemar, which sits in the same checklist as Chi-square, Kappa, and the relative risk and odds ratio options. SPSS restricts McNemar's test itself to 2×2 tables; for a paired table larger than 2×2, the software reports the McNemar-Bowker test of symmetry instead, under the same checkbox. The result lands in the Chi-Square Tests output table, in the row labelled "McNemar Test," carrying an Exact Sig. (2-sided) value. The common misread is grabbing the Pearson Chi-Square row instead, which sits in the same table if that box was also ticked out of habit, and which is testing the wrong hypothesis entirely for paired data.

Upload the dataset (or the relevant paired columns). StatsPlease's deterministic engine identifies the appropriate test from the variable type and study design, checks the relevant assumptions, computes the result using fixed, non-LLM algorithms, and drafts the Methods/Results sentence in AMA format, the same number a reader would get running the test by hand in SPSS.

References

1. McNemar Q. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika. 1947;12(2):153-157. https://doi.org/10.1007/BF02295996
2. Hoffman JI. The incorrect use of chi-square analysis for paired data. Clin Exp Immunol. 1976;24(1):227-229.
3. Fagerland MW, Lydersen S, Laake P. The McNemar test for binary matched-pairs data: mid-p and asymptotic are better than exact conditional. BMC Med Res Methodol. 2013;13:91. https://doi.org/10.1186/1471-2288-13-91
4. Bowker AH. A test for symmetry in contingency tables. J Am Stat Assoc. 1948;43(244):572-574. https://doi.org/10.1080/01621459.1948.10483284