Francis Galton, the pioneering statistician, was once at a village fair watching a competition to guess the weight of a dead ox. About 800 people took guesses and none of them was close to the mark. But Galton collected all the guesses and discovered the average of 543.4 kg was very close to the actual weight of 543.9 kg. This sparked off investigations by Galton into the wisdom of crowds. In one sense, every democratic election is a "wisdom of crowds" event. While individual voters may make decisions for odd reasons, political theorists believe democracies deliver more sensible leaderships on the whole than other forms of government. This is because, in aggregate, an electorate supposedly knows what suits it best.
Social scientists and intelligence agencies are both interested in more focussed tests. For example, the CIA funds an ongoing crowd-sourcing experiment - The Good Judgement Project - which is designed and run by three social scientists. Ordinary people, with no special expertise or access to information, are asked to make predictions about topical questions such as "How many Syrian refugees will exit the country?" or "Will Russia invade East Ukraine before May 10, 2014?" This experiment has shown promising results since the crowd-sourced predictions are apparently 30 per cent more accurate than estimates made by experts with access to far more information.
Certain wisdom of crowd effect is well-established in the stock market. Asset prices lead the real economy in both directions - the market rises and falls before economic cycles turn. Of course, the "madness of crowds" effect of bubbles and crashes is just as well-known.
All crowd-sourcing effects are amplified by the Internet. Stock market forums are a dime a dozen. One of the more interesting forums is the well-curated SeekingAlpha.com. This invites contributions by readers, which it selects and edits and publishes. It is a popular site and although it rarely has much material directly relevant to India, I read it regularly for news, views and ideas.
A study by researchers from City University, Hong Kong, Purdue University and The Georgia Institute of Technology says that the site has often beaten experts. The study compared over 100,000 articles (and comment on those articles) written between 2005-2012 to analysis by Wall Street analysts (highly qualified experts following specific stocks and sectors).
The study was looking at predictions of stock price direction and earnings surprises. The researchers created a "virtual portfolio" where they would go long on the most liked stocks and short the most disliked stocks on Seeking Alpha. This portfolio scored strong returns across the 2005-2012 period. It even delivered positive returns during the subprime crisis. This was way better than the Wall Street equivalents.
The site has much wider coverage than any Wall Street house, just as Wikipedia contains articles on far more subjects than any commissioned encyclopedia. The site's editorial board is also very good. Seeking Alpha publishes over 250 articles a day. The site also has good systems for identifying and compensating people, who have a good track record in terms of predictions.
India has its fair share of stock market sites and forums. Many of these are crude "pump and dump" operations and none are well-curated to my knowledge. Starting a copycat forum like a desi Seeking Alpha would also take considerable resources. Bandwidth and manpower would be required to build it and maintain it. Crowd-sourcing via an editorial board would also take a degree of skill. This sort of experiment would work only if it was very broad and had collated masses of data.
There is another intriguing point. Websites and forums of this nature are driven by retail. That's the "dumb money", while the analysts and institutions are the so-called "smart money". If the dumb money is collectively smarter than the smart money, then a basic market premise is wrong.
If a study like this is replicable across a wider array of sites, forums and markets, it's also more than likely the "smart money" will do its best to take over these information channels. In fact, this may already be happening in many cases. Every business realises these channels are useful and some are also trying to monetise their social media presence.
A smart operator, who scraped Indian market data and commentary off social media (Twitter and Facebook) and off comments at market-oriented websites, may be able to generate "liked/disliked" stocks, sectors and index categories. Such data could also be adjusted for known or apparent corporate biases. It would be fascinating to see whether the wisdom of crowds works better than the recommendations of analysts in the Indian markets.
