Every once in a while I get cranky about the excessive attention paid to bogosity. I shouldn't worry: in the long run, most silly pundits will simply be forgotten. And it's healthier to look for underappreciated good work than it is to fret about the wrongful kudos given to sloppy thinking. (see JustDesserts)

But nevertheless! Please forgive, for a moment, as I carp. Look at the stock market mavens, who every day have a different explanation (or the same explanation, spun differently) for why prices went up, or down, or stayed unchanged. Look at the social "scientists" who tell just-so stories purporting to predict cultural developments or interrelationships among groups. Look at the politicians who, depending on their audience, shift their positions like a weathervane in a whipping wind. (Look at me, as I ride my hobby horse off into the sunset ... or is that the sunrise?)

Imagine running an experiment: isolate a set of ersatz forecasters from the outside world and from each other. Feed them an identical subset of the day's newspapers: e.g., in the financial case give them all the non-business news. Then challenge them: which way did the markets move?

This could be extraordinarily revealing. Do the predictions agree with one another? With what really happened? How much better (or worse) are forecasts made with more (or less) data?

Meteorologists have long done these sorts of quantitative stress-tests of their weather models. So have biologists, chemists, physicists, and engineers of all sorts. It's not perfect; people still fool themselves in the "hard" sciences. But the incidence of silliness is somewhat lower, thanks to such discipline.

Ah, I feel so much better now...

TopicThinking - 2001-10-23

(correlates: HiStory, ReversionToTheMean, Refuse to Be Terrorized, ...)