Exit polls continue to overestimate odds of a close election
Exit polls, like the opinion polls before them, have overestimated the odds of a close finish in the Bihar assembly elections
Right from the outset, Election Metrics has maintained that the elections to the Bihar assembly would be hard to forecast. The big problem is the redrawing of alliances, with the Janata Dal (United), or JD(U), the Indian National Congress and the Rashtriya Janata Dal (RJD) coming together to form a Maha Ghatbandhan or Grand Alliance (GA).
The Bharatiya Janata Party (BJP), which fought the 2010 elections in the state together with the JD(U), has this time tied up with Ram Vilas Paswan’s Lok Janshakti Party (LJP) and Jitan Ram Manjhi’s Hindustani Awam Morcha (HAM) to form a strong alliance of its own.
As Election Metrics has mentioned several times in the past, there are two important steps to forecasting the outcome of elections in India. The first step is to determine the vote shares of different parties, which is usually done using a randomly sampled survey. The next step is to convert the thus obtained vote shares into seats, which is a rather complicated process given India’s first-past-the-post election system.
The problem with a massive realignment of alliances is that pollsters are deprived of a “prior model” in order to convert the vote share predictions to seats. In the absence of such a model, most pollsters have simply extrapolated from the vote share predictions to forecast seat distributions. And such extrapolation continues after the exit polls also.
Historically, exit polls have predicted election results in India more accurately than opinion polls. There are many reasons for this. Firstly, several people make up their minds on who to vote for only at the last minute. For this reason, even an opinion poll conducted a few days before the elections is unlikely to estimate vote shares accurately. Secondly, election turnouts in India have historically been of the order of 50-60%, and the turnout is not randomly distributed across various demographic factors. Thus, a pre-election opinion poll has the additional important step of forecasting turnout across demographic divisions, and predicting choices accordingly.
Exit polls have their downsides, too. As mathematician and psephologist Rajeeva L. Karandikar explained in the lead up to the 2014 general elections, the problem with exit polls is that random sampling suffers. Most opinion polls are conducted in homes of respondents who have been chosen following a scientific multi-level random sampling process. This ensures that the samples are representative and unbiased. Exit polls, on the other hand, are conducted outside polling booths, and given pattens in time of day of voting, etc.,. the sample is not as random, increasing chances of errors in forecasting.
Following the results of the 2014 general elections, Election Metrics had remarked that based on published vote and seat predictions, Today’s Chanakya was unlikely to survey in the same manner as other established polling agencies. There was a good chance, we had remarked, that Today’s Chanakya follows a Bayesian strategy when it comes to conducting surveys.
While the lack of a credible prior in Bihar (thanks to redrawing of alliances) might make Bayesian methods hard, the fact that Today’s Chanakya poll findings are different might explain why their vote share forecasts following the exit polls are far away from consensus.
The Today’s Chanakya exit poll predicts a seven percentage point difference between the BJP-led National Democratic Alliance (NDA) and the Grand Alliance, which they say will translate to the NDA gaining close to a two-thirds majority. It is the only pollster to come out with a decisive forecast.
Forecasts by others betray the lack of a prior model to convert votes to seats, as they have all forecast seat shares roughly proportional to their vote share forecasts (we had seen a similar case in the pre-election polls, and from this perspective, exit polls fail to add value). Matters are made worse by the fact that the vote share differences between the major alliances as per these pollsters is well within the margin of error, resulting in these polls adding next to no information.
The poll by India Today and Cicero has predicted a 1 percentage point advantage for the NDA, while that by India TV and C Voter has predicted a 1 percentage point edge for the GA. The poll conducted by ABP and Nielsen is marginally more decisive, giving a 2 percentage point advantage for the GA. However, with the margin of error for such polls being of the order of three percentage points, there is little to choose between the alliances based on any of these polls. As a consequence, these three pollsters have all taken a risk-free approach by predicting a close result.
The problem with forecasting a first-past-the-post election is that absolute vote shares don’t matter as much as the distribution of vote shares. Taking an extreme case, if the distribution of vote shares is absolutely consistent across constituencies, even a one vote advantage per constituency can lead to a clean sweep for the party. At the other extreme, the two alliances might have an identical overall vote share because one alliance has swept half the seats and the other alliance the other half, which leads to an assembly split right down the middle.
The reality is likely to be somewhere in the middle, but in the absence of a prior model (thanks to redrawing of alliances), a prediction is virtually impossible. There are techniques, of course, such as dividing the state by region and looking at vote share distribution in each region and making an educated guess, but by the looks of it none of the three pollsters mentioned above has gone down that path.
They cannot be blamed for having been conservative, however, since the incentives of pollsters are stacked that way. A standalone forecast that is horribly wrong can be highly damaging for a polling agency’s reputation. While there is still upside to being the only pollster to get forecasts right, the risk associated with such a strategy means that pollsters prefer to be with the crowd and wrong rather than sticking out and being possibly right.
In this context, there is no surprise that the established agencies have all come out with indecisive forecasts, predicting roughly equal vote shares for the two big alliances, and a roughly equally divided assembly. On the other hand, upstart Today’s Chanakya, which has less to lose in terms of reputation, has come out with a decisive forecast.
Based on early opinion polls, Election Metrics had commented that pollsters had overestimated the odds of a close election .
Looking at the results of the exit polls, there is no reason to revise this opinion. This doesn’t mean, however, that there will necessarily be a decisive victory for one of the alliances in the polls. All we are saying is that exit polls, like the opinion polls before them, have overestimated the odds of a close finish in the Bihar assembly elections.
Catch all the Politics News and Updates on Live Mint. Download The Mint News App to get Daily Market Updates & Live Business News.
MoreLess
First Published:6 Nov 2015, 04:57 PM IST
- Top Gainers
- Top Losers
- 52 Week High