Why I don't believe in The Wisdom of Crowds

In the Web 2.0 community The Wisdom of Crowds is held as an article of faith, but I personally do not subscribe to it, at least as practiced by the blogosphere. My rationale is primarily that too often these so-called "crowds" are in fact mobs, reacting in an irrational, knee-jerk manner, unwilling to let the dust settle before weighing in and more enamored of passion than of reason.

In fact, the Wikipedia article tells us about wise crowds and gives us the requirements for a "crowd" to show wisdom:

Not all crowds (groups) are wise. Consider, for example, mobs or crazed investors in a stock market bubble. Refer to Failures of crowd intelligence (below) for more examples of unwise crowds. According to Surowiecki, these key criteria separate wise crowds from irrational ones:

Diversity of opinion

Each person should have private information even if it's just an eccentric interpretation of the known facts.

Independence

People's opinions aren't determined by the opinions of those around them.

Decentralization

People are able to specialize and draw on local knowledge.

Aggregation

Some mechanism exists for turning private judgments into a collective decision.

If a crowd does exhibit these four key elements, then I would acknowledge the wisdom of the crowd, but so often (such as with the recent Digg Meltdown), multiple elements are missing. In fact, all too commonly, all we really see is a perversion of aggregation where a single opinion is amplified to the nth degree in the form of the infamous blogosphere echo chamber.

So, just because we have aggregated the voices of the crowd, does not automatically mean that the result will be wisdom. The Wikipedia article goes on to talk about "Failures of crowd intelligence":

Surowiecki studies situations (such as rational bubbles) in which the crowd produces very bad judgment, and argues that in these types of situations their cognition or cooperation failed because (in one way or another) the members of the crowd were too conscious of the opinions of others and began to emulate each other and conform rather than think differently. Although he gives experimental details of crowds collectively swayed by a persuasive speaker, he says that the main reason that groups of people intellectually conform is that the system for making decisions has a systematic flaw.

Surowiecki asserts that what happens when the decision making environment is not set up to accept the crowd, is that the benefits of individual judgments and private information are lost, and that the crowd can only do as well as its smartest member, rather than perform better (as he shows is otherwise possible). Detailed case histories of such failures include:

Too homogenous

Surowiecki stresses the need for diversity within a crowd to ensure enough variance in approach, thought process, and private information.

Too centralized

The Columbia shuttle disaster, which he blames on a hierarchical NASA management bureaucracy that was totally closed to the wisdom of low-level engineers.

Too divided

The U.S. Intelligence community failed to prevent the September 11, 2001 attacks partly because information held by one subdivision was not accessible by another. Surowiecki's argument is that crowds (of intelligence analysts in this case) work best when they choose for themselves what to work on and what information they need. (He cites the SARS-virus isolation as an example in which the free flow of data enabled laboratories around the world to coordinate research without a central point of control.)

Recent reports indicate that the CIA is now planning a Wikipedia style information sharing network that will help the free flow of information to prevent such failures again.

Too imitative

Where choices are visible and made in sequence, an "information cascade" can form in which only the first few decision makers gain anything by contemplating the choices available: once this has happened it is more efficient for everyone else to simply copy those around them.

Too emotional

Emotional factors, such as a feeling of belonging, can lead to peer pressure, herd instinct, and in extreme cases collective hysteria.

Wow, this puts the whole Digg Meltdown in a different light.

My own personal has been what I call "the sum of all curves." Each of us has our own views which we express and it is by expressing all of these views independently and looking at the net views (such as by taking polls or observing behavior in markets), we can get our collective views as the sum of all of the curves for each topic of interest. Whether talking about the stock market, political issues, or general social issues, "markets" work best when we each develop our own views independently (even if we all look at the same, shared information, but hopefully including information that we each discover on our own) and then use a "market" (or poll) to calculate the sum of all curves to produce the net curve (or set of curves).

I also believe that we need to forcefully challenge the notion that measuring raw popularity somehow implicitly gives us a measure of authority or even relevance.

In summary, I do in fact believe in The Wisdom of Crowds in its theoretical form as put forward by James Surowiecki, but I do not subscribe to it as currently practiced by the Web 2.0 community.

-- Jack Krupansky