Via Economist.com

Perhaps the most compelling reason why market prices are tough to beat is the “wisdom of crowds” phenomenon. If people are asked to estimate the number of jellybeans in a jar, their average estimate is usually quite close to the truth; indeed the average guess is far better than the vast majority of individual guesses. In other words, as Michael Mauboussin of the fund-management group Legg Mason remarks, the collective is smarter than the average person within the collective.

But this wisdom depends on the diversity of the people making the guesses. Mr Mauboussin argues that problems occur when diversity breaks down and “groupthink” starts to take over. Investors no longer guess how many jellybeans are in the jar, but what other people’s guesses might turn out to be.

There’s an old test that neatly makes this point. Participants have to choose a number between zero and 100 that will be two-thirds of the average choice of the others taking part. So if you thought the average would be 50, you would go for 33. However, if everyone makes this logical leap, the best guess should be 22 (two-thirds of 33). Extend this process a few times and you can work out that the best choice would be zero. In real life, however, not everyone is so rational and the correct answer is never that low.

In short, it is very hard to quantify the precise irrationality of investors. That is why investors get lured into buying dotcom stocks in the hope that a “greater fool” will purchase them at a higher price. And that is why there will always be market anomalies for academics to discover.