Are exit polls reliable?

Jairam Ramesh
Joint Secretary,
Economic Affairs, AICC

For political parties that conduct their own version of exit polls by asking their workers for political feedback after an election, polls conducted by agencies have to be factored into the campaign whether they are reliable or not.

The Bharatiya Janata Party-National Democratic Alliance (BJP-NDA) certainly thinks they are reliable. Look at their behaviour. The opinion polls after March 26 predicted a sweep for the NDA and forecast that Congress would dip below 100.

BJP President Venkaiah Naidu immediately said the Congress should take vanvas and that the NDA would get 300 seats. When the first phase of polling was over, the prime minister took a highly-publicised review meeting that was attended by the BJP brass.

Then came the second phase of the elections and the exit polls in which a Congress victory was forecast. The same Venkaiah Naidu blasted the exit polls and called them cheerleaders of the Congress! There were no review meetings, no tall claims by Pramod Mahajan and Arun Jaitley. No one from the BJP said a word.

Our party, too, reacted to the exit polls. After the first set of opinion polls, it was tough to get the party out of the despondency into which it had fallen. It was hard to get people to appear on TV for debates.

But after the exit polls on April 26, I didn't have to call anyone in the party "" they called me! They wanted to know if the party was holding any meetings, where they were being held, they wanted to advise the party and help it... So while on March 28 I was trying to boost the party spirit and tell them that the exit polls are not the final test to predict victory or defeat, on April 26, I was trying to tone people down and tell them we should not become complacent.

My sense is that however much we may deny it, exit polls are seen as reliable even if they're not and they affect the sentiment and sensibility of political parties.

There is no way you can check if an exit poll is accurate or not because the situation in which you interview your respondent can never be replicated. It's an online poll so you can't eliminate biases. By contrast, an opinion poll might yield different results at different times and, therefore, can be corrected for biases. An exit poll, on the other hand, can be either totally wrong or absolutely accurate.

The first exit poll was done in 1989. I was part of that exercise. Systems have been refined since then. I must point out that our strategy is not fashioned only on the basis of results of exit polls. We get feedback from our own people and form a general assessment.

For instance, the Aaj Tak exit poll in Gujarat gives us just five seats. Our own assessment is that we will get seven or eight, based on a variety of factors. But strategy is formulated on the basis of several other factors.

For instance, for us, the May 10 election was not a make or break election. But May 5 was, because in Madhya Pradesh, Rajasthan and Uttar Pradesh, we had only a few seats with which to improve the tally. The issue was whether we should have analysed the May 5 election on the basis of the last voting exercise the people had undergone, that is, assembly elections; or whether we should treat the 1999 voting pattern as the benchmark for the way people will vote this time.

According to the 1999 results, we should have picked up 15 to 20 seats in Madhya Pradesh and Rajasthan. But according to the 2003 results, we should be getting zero seats. Even the exit polls can't tell you with accuracy what the result will be. So, it is the political assessment that is important.

But what the exit polls have proved is that the NDA is not going to reach 300 seats as predicted by Venkaiah Naidu. And the Congress is not going to be reduced to double digits as Deputy Prime Minister L K Advani had predicted.

Because of this, our workers are putting in the last-mile effort, going on the assumption that they are reliable. Thus, exit polls do make a difference, whether reliable or not.

(As told to Aditi Phadnis)

Santosh Desai
President
McCann Erickson

Either something is reliable or it is not. Reliability is clear in its binariness "" you cannot be somewhat or sometimes reliable. So, are exit polls reliable? I am not a market researcher, but it would appear that for any instrument to be reliable, it must be both consistently accurate and replicable.

Without going into deep historical analysis, it is possible to conclude that exit polls falter on both fronts. While there have been some notable successes, there have been more than a few failures "" some of them quite spectacular. Take the 2003 assembly elections where almost no one predicted the BJP wins in Madhya Pradesh and Rajasthan.

The trouble is that with an exit poll, we expect high standards of accuracy because it occurs after the event "" all the uncertainty that surrounds pre-poll surveys stands eliminated. Barring the possibility that a voter has lied about whom she voted for, errors in predictions are entirely because of the polling technique.

The fact that different market research agencies "" all reputed ones "" come up with different estimates is good evidence that exit polls are not the precise science they profess to be. How reliable can an instrument be if a different reading comes up every time someone else uses it?

As an example, take a look at the exit poll predictions that followed the first phase of elections for 140 seats. For this phase, Zee News-Taleem gives the NDA between 69 and 73 seats while Aaj Tak-ORG gives it 93.

Even more interestingly, Zee gives other parties between 23 and 29 seats while Aaj Tak's prediction is a measly three seats! And this is by no means an exceptional case "" exit polls predictions often vary significantly.

Some would argue that the problem lies with the proliferation of exit polls without sufficient attention being paid to quality. After all, given our highly complex social reality with its intricate caste and community equations, getting a representative sample is not the easiest of tasks.

Also, our first-past-the-post system ensures that even a small error in converting vote shares into seats can have major repercussions on the final tally. Getting it right, therefore, depends a lot on the expertise and experience of the pollsters involved. So, is it just a matter of listening to the right surveys and ignoring the others?

The trouble is in figuring out which are the right ones. There isn't a clear cut divide between the good surveys and the bad. Don't forget that no one got the 2003 Assembly predictions right.

The bottomline is that exit polls have not delivered accurate predictions often enough to be deemed reliable. So what explains their popularity? Why are our TV channels crammed wall-to-wall with exit polls?

I would argue that what makes exit polls tick is the spectacle generated by the suspicion of its unreliability. The exit poll appears to be scientific enough to merit consideration and fallible enough to feed speculation.

The exit poll is interesting because it trembles with accuracy "" the very preciseness of its prediction is an open invitation to its own possible demise. The exit poll crystallises all speculation into a single number, which in turn is endlessly speculated upon. The role of the exit poll is not to close speculation, but to multiply it.

Exit polls today need to be situated in the entertainment industry, and not in market research. What produces entertainment is the friction caused by the differences in prediction across channels, and the apparently-legitimate indignation of those who are shown to be losing.

The entire army of analysts, commentators, election pundits and interpreters derive more material to tiptoe around, more sounds to bite into, and more TV minutes to fill up. If the exit polls were really reliable, it would merely substitute the final result. There would be little incentive for so many different polls to co-exist; one definitive poll would suffice.

In media capitalism, any asset that generates media time and space is invaluable. Exit polls are not reliable, but make no mistake, they are extremely useful. They give us a lot to talk about, and in this case, something to write about.