Do not expect the unexpected
If you call up a random American and ask them to guess the demographics of America, here is what they’ll tell you, on average:
- ~33% of Americans are black1,2 (~3x the actual of ~13%3)
- ~25% of Americans are gay or lesbian4 (~5x the actual of 5%5)
- ~15% of Americans are Muslim (~15x the actual of 1%)6,7
- ~11% of Americans are Jewish 2 (~10x the actual of ~1%-2%8)
If you ask what Americans are likely to die of, they’ll tell you, on average:
- ~8% of Americans are killed by murder (actual is ~0.7%)9
- ~4% of Americans are killed by STIs (actual is ~0.3%)9
- ~4% of Americans are killed by terrorism & conflict (actual is ~0%)9
And if you ask about corporations and government, they’ll tell you, on average:
- Companies’ profit margins are ~36% (~5x the actual of ~7%)10
- The federal budget for foreign aid is ~28% (actual is ~1%)11,12,13
From these guesses, what should you conclude?
- That people are ignorant? (Probably. Half of Americans believe in alien UFOs,14,15 ghosts,16 and a young Earth,17 after all.)
- That people don’t take surveys seriously? (Probably. Scott Alexander says the Lizardman’s constant is 4%.)
- That people don’t give accurate answers without an incentive? (Probably. Prior et al. found that cash incentives improved partisans’ accuracy at guessing economic conditions by 5%abs.)
Here is what I conclude:
If you ask people to estimate a low percentage that they don’t know, they’re going to overestimate it. Maybe they’re hedging toward 50%. Maybe it’s easier to imagine the specific thing being estimated than the amorphous remainder. Maybe rare events are amplified by a media system that feeds our desire for novelty, originating from an evolutionary environment where we had to learn about dangers without sufficient statistics. Or maybe it’s just the fact that if you guess a random percentage, it’s more likely to pull the average toward 50% than away.
Whatever the reason, remember that rare events are usually overestimated. Do not expect the unexpected.18
Footnotes
-
Kieran Healy reporting on the General Social Survey (2000) ↩ ↩2
-
AEI reporting on data from Reason-Rupe (2013) and Yahoo!Finance ↩
-
Over many polls, something like 36%–62% of Americans profess to believe in ghosts. ↩
-
In prediction markets and other aggregators of judgment, Tetlock and others have found that aggregated predictions are usually hedged toward 50% and can be improved by a bluntly ‘extremizing’ them away from 50% and toward 0% or 100%. Here’s a paper by Baron et al discussing two reasons why it makes sense to extremize aggregated beliefs. And here’s a reveiew of Superforecasting by Jason Collins that mentions the topic. ↩