Real Examples of Misleading Statistics and How to Spot Them

A statistic can be correct and still create a false impression. The problem is often not the arithmetic, but missing context: what was counted, who was included, which groups were combined, or how the result was drawn on a graph.

Here are concrete examples of common traps, including a well-known historical case. The numerical illustrations are deliberately simple so you can check each one yourself.

Relative risk can hide the absolute difference

Suppose a report says a treatment increases an unwanted outcome by 50%. That sounds substantial, but the starting risk matters.

Imagine that, over the same follow-up period, the outcome occurs in 2 of every 10,000 people without the treatment and 3 of every 10,000 with it:

• Without treatment: $2/10{,}000 = 0.02\%$.
• With treatment: $3/10{,}000 = 0.03\%$.
• Relative increase: $(3-2)/2 = 50\%$.
• Absolute increase: $3/10{,}000 - 2/10{,}000 = 1/10{,}000$, or $0.01$ percentage points.

Both “50% higher” and “one additional case per 10,000 people” describe the same difference. The first emphasizes the relative change; the second makes the change in individual probability clearer. To assess a real study, also check the number of participants, the follow-up period, and whether the difference is uncertain. A small observed difference is not automatically evidence of a reliable effect.

Averages can hide different groups

In 1973, the University of California, Berkeley, reported that about 44% of male applicants and 35% of female applicants were admitted to graduate programs. Looking only at those totals suggests a large overall gap. But applicants did not apply in the same proportions to each department: some departments admitted a much larger share of applicants than others, and women were more likely to apply to departments with lower admission rates. The department-level analysis changed how the aggregate gap should be interpreted. The case was reported by Bickel, Hammel, and O’Connell in “Sex Bias in Graduate Admissions: Data from Berkeley.”

A small made-up table shows how aggregation can even reverse a comparison:

Department	Men admitted	Men’s rate	Women admitted	Women’s rate
A	90 of 100	90%	19 of 20	95%
B	2 of 10	20%	3 of 10	30%
Total	92 of 110	83.6%	22 of 30	73.3%

Women have a higher admission rate in each department, but a lower rate overall. That happens because most male applicants in this example applied to the high-admission department A, while a larger share of female applicants applied to B. This is an instance of Simpson’s paradox: a trend in combined data can differ from the trends within subgroups.

The Berkeley analysis does not by itself prove that every department or the wider system was fair. Department choice may reflect other influences, and an aggregate comparison cannot answer every causal question. It does show why totals should be examined alongside relevant subgroups.

A graph’s axis can exaggerate a difference

Suppose two bars represent values of 50 and 55. The second value is 10% higher than the first. If the graph’s vertical axis starts at 49 rather than zero, the bars have visible heights of 1 and 6 units. The second bar therefore looks six times as tall, even though the underlying values differ by only 5.

A truncated axis is not always wrong: line charts often use a narrower range to reveal small changes, and the axis may be clearly labeled. But for bar charts, where bar length invites direct comparison, a zero baseline is usually easier to interpret. Read the axis labels before judging the size of a visual difference.

A selected sample can omit the important cases

A classic World War II example concerns analysis of bullet damage on aircraft that returned from missions. The returning planes showed damage in some areas and little damage in others. If you study only those aircraft, you see only planes that survived. A lack of bullet holes in a particular area may mean that hits there were especially likely to prevent a plane from returning—not that the area was safe.

This is survivorship bias: the observed sample excludes cases that did not make it into the data. Abraham Wald’s wartime analysis of aircraft vulnerability is a well-known example. The same logic applies to reports about successful businesses, patients who completed a program, or products still available for sale. Ask which cases could have disappeared before measurement.

A quick checklist for evaluating a statistic

Before accepting a headline or chart, ask:

1. What is the denominator? A percentage without its count or comparison group can be hard to interpret.
2. Are the groups comparable? Check age, risk level, location, time period, and other relevant differences.
3. What is the average hiding? A mean can be pulled by a few extreme values; medians and distributions may add context.
4. Who is missing? Look for nonresponse, dropouts, people excluded by the study design, or cases that did not survive to be measured.
5. How uncertain is the estimate? Sample size and uncertainty matter, especially when comparing close results.
6. Does the graphic match the claim? Check axis limits, units, whether the scale is linear, and whether the display uses counts or rates.

Common mistakes

• Treating a relative percentage as though it were an absolute change.
• Comparing totals when the groups have different compositions.
• Assuming that a statistically reported difference is large or practically important.
• Treating a graph as deceptive just because its axis does not start at zero; instead, judge whether its scale is clearly labeled and suitable for the display.
• Assuming a misleading presentation proves deliberate deception. Sometimes context was omitted by mistake, or the statistic answers a narrower question than readers assume.

A useful habit is to translate headline figures back into counts, inspect the groups behind a total, and read the data-collection method before deciding what a result means.