Infotopia by Cass Sunstein is subtitled "How Many Minds Produce Knowledge". That subtitle really should add "(or Fail to Do So)" since large portions of the book are devoted to problems of opinion surveys, group deliberation, and related mechanisms for adding together individual judgments. Infotopia reads like a legal brief, perhaps understandably: Sunstein is a Harvard law professor and currently serves as a senior US Government official. It's heavily-footnoted and somewhat slow-paced, often pausing to restate the obvious and then turning to "on the other hand" exceptions. It's also unfortunately repetitive. I lost count of references to the Condorcet Jury Theorem, the Challenger disaster, Friedrich Hayek, and the Iraq WMD case; each is mentioned over a dozen times. Surely there are more diverse examples worth citing?

But for those (Harvard law professors?) who are relatively new to wikis, blogs, prediction markets, and suchlike new developments, Infotopia provides a decent primer on the pitfalls and potentials. In the Introduction Sunstein categorizes four ways that a group could combine information:

• statistical averaging of independent judgments by members;
• deliberation, "the reasoned exchange of facts, ideas, and opinions", possibly followed by voting (the jury system);
• market mechanisms that allow participants to "buy and sell on the basis of their judgments"; and
• "enlist the Internet" to implement variations on the above, including "massive surveys, deliberative forums, prediction markets, books and resources that anyone can edit, ...".

Each of these is analyzed at length. Prediction markets come out well, when there's information to be aggregated and when manipulation or speculative bubbles aren't a threat. Averaging can work too, though the "wisdom of crowds" is often overrated. Deliberation frequently fails, according to the author: information isn't shared, some members dominate discussion while others succumb to social pressures to conform, and early errors tend to be amplified until they become locked-in. Wikis and open-source software development are often surprisingly productive. Blogs, in contrast, provide "... a stunning range of claims, perspectives, rants, insights, lies, facts, falsehood, sense, and nonsense."

Bottom line? Sunstein is optimistic, but as always with a caveat. "Far more than ever before," he concludes, "humanity has promising methods for seeking out widely dispersed information and creativity and for aggregating these into uncannily productive wholes. The ultimate value of the new methods depends, of course, on how we use them."

Hard to disagree with that!

^z - 2010-06-13