Comrade James Blodgett (with whom in 1978 I wrote my first article for Byte magazine — cf. PetBibli1 (2000-05-23)) is troubled, perhaps justifiably, about the risks of particle accelerator experiments which might create tiny but world-destroying black holes. I'm rather skeptical of this threat to our civilization, for multiple reasons (see http://www.risk-evaluation-forum.org/, where many arguments and counter-arguments are discussed). And I'm far more fretful about some high-probability high-impact challenges that loom—such as rampant human ignorance and the not-unrelated destruction of complex natural systems. More on that another time.
Meanwhile, thinking about low-probability high-impact phenomena raises the fascinating mathematical question: how may wildly divergent likelihood estimates best be combined?
Consider a trivial example: what's the chance of dealing four aces from the top of a normal shuffled deck of cards? (No fair computing the exact answer until after you make your own guess!) Suppose you survey four people and get four off-the-cuff estimates: 10-3, 10-4, 10-5, and 10-6. What's the best way to reconcile numbers that vary by orders of magnitude?
The simple arithmetic mean gives 2.8 * 10-4. But such averaging is pretty clearly a bad approach. It places entirely too much weight on a single person's opinion: whoever makes the highest guess. And if that person is wildly off-base, has an axe to grind, or otherwise wants to swing the results, s/he can certainly do so.
A better approach might be to avoid most of the math and just take the median of the estimates, the middle of the sorted list. For our toy problem that median is 5.5 * 10-5, halfway between the second and third figures. Medians are insensitive to out-of-the-ballpark extremes, since all that matters are the rank-ordering of the numbers and the value(s) of the central one(s).
But shouldn't high or low judgments have at least some non-zero influence on the outcome? That philosophy leads to my personal favorite technique: take the logarithms of the probabilities, average them, and then exponentiate. The exponential-mean-log in our example is 3.2 * 10-5, not far from the median answer. In other cases, however, an E-M-L might give a better unified group judgment.
(That is, I speculate that human estimation for rare events has an error distribution that tends to be logarithmic. True? And if anyone cares: the actual chance of dealing four aces = (4*3*2*1) / (52*51*50*49) = 3.7 * 10-6. Cf. BiggerPictures (1999-11-22), MortalityFunctions (2000-10-30), RootMeanSquareDance (2004-04-24), ...)
TopicScience - TopicSociety - 2005-05-31
(correlates: LastManStanding, FirmwareBugs, ReadyWillingAble, ...)