Test selection & reporting · ~6 min read
Chi-Square and Fisher's Exact
The same 2×2 table needs different arithmetic depending on how small your cells are.
Published
In short
Chi-square and Fisher's exact test the same kind of question, whether two categorical variables are related, but they get there by different arithmetic, and only one of them is valid when a contingency table has small expected cell counts. The rule of thumb, every expected count at least 5, decides which one you're allowed to report. Below is a real result computed both ways on trial data, and a worked case showing where the two tests actually stop agreeing.
Same question, two different arithmetic paths
Chi-square works by approximation: it compares the counts you observed in a contingency table to the counts you'd expect if the two variables were unrelated, and uses a distribution that only holds up well when the expected counts in every cell are large enough. Fisher's exact test doesn't approximate anything. It calculates the exact probability of the observed table, and every more extreme one, directly from the hypergeometric distribution, which is why it has no minimum-count requirement at all.1
The convention almost every stats package still uses, expected counts of at least 5 in every cell, traces to Cochran's 1954 recommendation for when the chi-square approximation becomes reliable.2 Below that, chi-square's P value can't be trusted, and Fisher's exact is the correct choice regardless of how the rest of the analysis is set up.
A real result: does treatment affect mortality?
The Mayo Clinic's primary biliary cirrhosis (PBC) trial randomised 312 patients to D-penicillamine or placebo.3 At last follow-up, 65 of 158 patients on D-penicillamine had died, compared with 60 of 154 on placebo.
| Group | Died | Alive |
|---|---|---|
| D-penicillamine | 65 | 93 |
| Placebo | 60 | 94 |
Expected counts (94.7, 63.3, 92.3, 61.7) are all comfortably above 5, so the standard chi-square is valid here.
χ2(1) = 0.08 · P = .782 · OR = 1.10 (95% CI, 0.70–1.72)
Mortality did not differ significantly between the D-penicillamine and placebo groups (χ2[1] = 0.08, P = .782; odds ratio for death with D-penicillamine, 1.10; 95% CI, 0.70–1.72).
Figures computed with scipy from real trial data (PBC trial, Mayo Clinic).
Example data: Mayo Clinic primary biliary cirrhosis (PBC) trial (Fleming and Harrington, 1991); see references. Computed with scipy, not generated.
Fisher's exact test on the same table gives P = .730, close enough to the chi-square P value that it changes nothing about the conclusion. That agreement is itself worth noticing: when expected counts are adequate, the two tests will almost always land in the same place. The real reason to know both isn't that they disagree here. It's that a smaller series won't have this luxury.
Where the two tests actually stop agreeing
Small numbers are the normal case in surgical case series, not the exception, and that's exactly where the two tests diverge. Suppose a single-centre series reports a rare postoperative complication in 3 of 14 patients who received one closure technique, against 0 of 12 who received another.
| Complication | No complication | |
|---|---|---|
| Technique A | 3 | 11 |
| Technique B | 0 | 12 |
The expected count in the "Technique B, complication" cell here is under 2, well below the threshold. Chi-square on this table returns an approximate P value that Cochran's own rule says not to trust. Fisher's exact test, run on the same table, gives the exact answer directly, P = .22, without needing that approximation to hold at all.
Writing the AMA sentence
Name the test you actually used and why. If you used Fisher's exact because a cell's expected count fell under 5, that's worth stating explicitly, since a reviewer checking your table should reach the same conclusion you did about which test the data required. Report the odds ratio with its confidence interval alongside either test.
Try it yourself
Reproduce this result
The example above was computed from a public dataset. Run it in either tool and the numbers match, because both are doing the same computation, not generating it.
In StatsPlease
- Upload the dataset as a CSV.
- Open the Analysis tab.
- Select the two categorical variables: vital status and treatment.
- Press the Category Patterns preset button. There is no separate run step; pressing the preset runs the analysis, checking expected cell counts and routing to chi-square or Fisher's exact.
- On the Results tab, the expected-cell check, the test chosen, and the AMA sentence are shown together.
In SPSS
- Analyze → Descriptive Statistics → Crosstabs, request expected counts.
- Check whether any expected count falls below 5.
- Analyze → Descriptive Statistics → Crosstabs → Statistics → Chi-square (and Fisher's exact, available by default for 2x2 tables).
- Read whichever row the expected-count check actually supports.
- Compute the odds ratio and its CI separately; SPSS does not report it by default.
Frequently asked questions
When should Fisher's exact test be used instead of chi-square?
When any cell in the contingency table has an expected count below 5. Below that threshold, chi-square's approximation becomes unreliable, and Fisher's exact calculates the exact probability directly without needing that approximation to hold.
Do chi-square and Fisher's exact usually give different results?
Not when expected counts are adequate; the two tests typically agree closely. They diverge specifically in small samples with low expected cell counts, which is common in single-centre surgical case series.
Should an odds ratio always be reported alongside a chi-square or Fisher's exact result?
Yes. A P value alone indicates whether an association was detected but says nothing about its size or precision; the odds ratio with its confidence interval provides both.
You might also read
References
- Fisher RA. On the interpretation of χ² from contingency tables, and the calculation of P. Journal of the Royal Statistical Society. 1922;85(1):87-94. https://doi.org/10.2307/2340521
- Cochran WG. Some methods for strengthening the common χ² tests. Biometrics. 1954;10(4):417-451. https://doi.org/10.2307/3001616
- Fleming TR, Harrington DP. Counting Processes and Survival Analysis. New York: John Wiley and Sons; 1991.
StatsPlease checks expected cell counts on your own uploaded table before choosing between chi-square and Fisher's exact, so the test named in your results tab is the one the data actually supported. Select your two categorical variables and press the Category Patterns preset; pressing it runs the analysis.
Try StatsPlease free