Simpson's Paradox

Simpson’s Paradox happens when a trend appears in several separate groups but reverses or disappears when the groups are combined. This usually happens because a hidden factor changes the relative sizes or makeup of the groups, making the overall average misleading.

Below are two groups of students, each showing a positive relationship between study hours and test scores: more studying means better scores within each group. But watch what happens when the two groups are combined into one: the overall trend reverses. None of the underlying data changes, only how it's grouped. Drag the slider slowly from left to right to watch the story unfold.

This scenario is a standard textbook way of demonstrating Simpson's Paradox with made-up data. The same reversal shows up in real studies too. For example, in a 1986 British Medical Journal comparison of kidney stone treatments, Treatment A outperformed Treatment B on both small stones (93% vs. 87% success) and large stones (73% vs. 69% success), yet Treatment B appeared to win overall (83% vs. 78%) once the case mix was ignored. Further reading: Simpson's Paradox on Wikipedia.