The recent book Superforecasting: The Art and Science of Prediction by Philip Tetlock and Dan Gardner is fun, fast, and insightful. It's a chatty expansion of the technical articles that Tetlock and colleagues have recently published in scientific journals, based on their work to identify the factors that make for better human prediction of events. Parts are a bit heavy-handed in their self-help second-person you-should-do-this style; parts are too personal-anecdotal; and parts gloss over important quantitative issues. But overall, Superforecasting is a worthy read.

The list "Characteristics of Superforecasters" from the end of Chapter 8 ("Perpetual Beta") summarizes many of their key discoveries. Another useful list appears in the Appendix ("Ten Commandments for Aspiring Superforecasters"):

• Triage - "Focus on questions where your hard work is likely to pay off. Don't waste time either on easy 'clocklike' questions (where simple rules of thumb can get you close to the right answer) or on impenetrable 'cloud-like' questions (where even fancy statistical models can't beat the dart-throwing chimp) ..."
• Break seemingly intractable problems into tractable sub-problems - "... Decompose the problem into its knowable and unknowable parts. Flush ignorance into the open. Expose and examine your assumptions. Dare to be wrong by making your best guesses ..."
• Strike the right balance between inside and outside views - "... uniqueness is a matter of degree ... conduct creative searches for comparison classes even for seemingly unique events ... pose the outside-view question: How often do things of this sort happen in situations of this sort? ..."
• Strike the right balance between under- and overreacting to evidence - "Belief updating is to good forecasting as brushing and flossing are to good dental hygiene. ... Skillful updating requires teasing subtle signals from noisy news flows—all the while resisting the lure of wishful thinking ..."
• Look for the clashing causal forces at work in each problem - "... In dragonfly eye, one view meets another and another and another—all of which must be synthesized into a single image ..."
• Strive to distinguish as many degrees of doubt as the problem permits but no more - "... Nuance matters. The more degrees of uncertainty you can distinguish, the better ..."
• Strike the right balance between under- and overconfidence, between prudence and decisiveness - "... understand the risks both of rushing to judgment and of dawdling too long near 'maybe' ... find creative ways to tamp down both types of forecasting errors—misses and false alarms—to the degree a fickle world permits such uncontroversial improvements in accuracy ..."
• Look for errors behind your biases but beware of rearview-mirror hindsight biases - "... although the more common error is to learn too little from failure and to overlook flaws in your basic assumptions, it is also possible to learn too much (you may have been basically on the right track but made a minor technical mistake that had big ramifications) ... do postmortems on your successes too ..."
• Bring out the best in others and let others bring out the best in you - "... Master the fine arts of team management, especially perspective taking (understanding the arguments of the other side so well that you can reproduce them to the other's satisfaction), precision questioning (helping others to clarify their arguments so they are not misunderstood), and constructive confrontation (learning to disagree without being disagreeable) ..."
• Master the error-balancing bicycle - "... implementing each commandment requires balancing opposing errors ... Learning requires doing, with good feedback ... super forecasting is the product of deep, deliberative practice ..."
• Don't treat commandments as commandments - "'It is impossible to lay down binding rules', Helmuth von Moltke warned, 'because two cases will never be exactly the same.' ... Guidelines are the best we can do in a world where nothing is certain or exactly repeatable. Super forecasting requires constant mindfulness ..."

Yep, it's all about mindfulness, awareness, metacognition ... and also openness and optimism. From Chapter 5 ("Supersmart?"):

... In personality psychology, one of the "Big Five" traits is "openness to experience," which has various dimensions, including preference for variety and intellectual curiosity. It's unmistakable in many super forecasters. Most people who are not from Ghana would find a question like "Who will win the presidential election in Ghana?" pointless. They wouldn't know where to start, or why to bother. But when I put that hypothetical question to Doug Lorch and asked for his reaction, he simply said, "Well, here's an opportunity to learn something about Ghana."

(cf. Solve the Problem (2007-05-24), ...) - ^z - 2016-02-21