Poll fault

British poll failures are applicable to finance

David Cameron's swift return to 10 Downing Street last week triggered the deepest blushes for UK opinion pollsters in more than two decades. The most likely culprits - chiefly groupthink and over-reliance on models - are the same ones that tend to fuel financial conflagrations, too.

On the eve of Thursday's general election, essentially all the opinion polls had the Conservatives and Labour Party neck-and-neck, neither anywhere near a majority in the 650-seat House of Commons. That was turned on its head by a TV-commissioned exit poll, revealed after voting ended, which predicted 316 seats for the Tories, much closer to an outright victory than expected, and 239 for Labour. The final result - 331 to 232 - was even more extreme.
 
One likely flaw in the polling was a herd mentality. Damian Lyons Lowe, founder of research firm Survation, says it "chickened out" of publishing figures showing a big Conservative advantage based on telephone polling last Wednesday. The results just seemed too "out of line". But even wacky outputs shouldn't be dismissed. One related concern in finance is the idea that many fund managers would rather fall short of industry benchmarks together than take a risk to outperform alone.

Survation's poll may fit with a very late swing to the Tories. In hindsight, though, questions have been raised about discrepancies between telephone and online polling, "shy Conservatives" unwilling to declare their voting intention, and other potential explanations. YouGov's Anthony Wells notes the need to review how researchers chose their samples and assessed respondents' likelihood of actually voting (one thing exit pollsters don't have to worry about).

If an industry investigation fingers these areas, it suggests insufficient scepticism about the models that turn polling data into national projections. Finance throws up examples, too: more critical scrutiny of the rating agencies' data on mortgage default by US subprime borrowers might have helped limit the lending excesses that led to the 2008 financial crisis. And too often, numerical outputs from complex models, like value-at-risk figures, are treated as definitive by banks and regulators.

Even Nate Silver, the near-perfect seer of the 2008 US presidential election, badly missed the UK result. Silver's FiveThirtyEight colleague Ben Lauderdale suggests confidence limits on polling results were set too tight given the experience of 1992 and the atypically large number of political parties meaningfully involved last week. In other words, the Conservative win suggests a fatter tail on the distribution of results than was assumed. That's the kind of language financiers understand.