People don't tend to publicize their failures. Scientists keep quiet about experiments that don't work. (What journal would want to print papers about non-events?) Nobody likes to risk looking foolish by talking about something that might be a mistake. That's human nature.
But the result is an unintentional bias in the literature — and that slants results. Researchers who are less than honest, or who observe a statistical fluctuation, get the headlines. "Expert witnesses" are paid to take one side of an argument, and cheerfully do so. The more articulate (or noisy, or photogenic) a pundit, the likelier that the media will favor that side of the technical debate.
So beware reported "breakthroughs" until they have been verified by neutral parties. Cold fusion? Polywater? A host of front-page "discoveries" in medicine, sociology, and psychology? Take with a grain of salt. Most are honest mistakes; some are deliberate frauds; a few will turn out to be important and are worth studying, after confirmation. Apply extra skepticism to proofs or disproofs of cherished beliefs, religious or social, where unconscious bias is greatest.
Tuesday, November 02, 1999 at 21:02:58 (EST) = 1999-11-02
(correlates: ThinkingThroughPrejudice, LearningInvestment, No Concepts At All, ...)