Bihar results: why did the exit polls get it wrong?
It’s now a well established fact that opinion and exit polling agencies had a horrid time in the recently concluded Bihar assembly elections.
Not only did the published polls fail to predict the magnitude of victory by chief minister Nitish Kumar’s Grand Alliance in terms of seat share, but none of the pollsters got the vote share right either.
While Today’s Chanakya predicted that the Bharatiya Janata Party-led National Democratic Alliance (BJP-led NDA) would get 46% of the votes and the Grand Alliance 39%, most of the other pollsters had given both alliances about 40% of the votes each.
The notable exception was Axis, which correctly predicted that the Grand Alliance would win by a landslide, but their poll was curiously withdrawn by the television news channel CNN-IBN, which had commissioned it. But Axis, too, could not get the vote shares right, overshooting its estimates for both alliances by a few percentage points.
We will start the analysis by looking at the overall vote share won by each party in the recent assembly elections and comparing them with the overall vote share in last year’s general elections in Bihar. Given the change in the proportion of seats contested by each party in each election, this comparison isn’t fully correct, but is a good starting point.
Figure 1 shows a scatter plot of the vote share of each party in the 2014 and 2015 elections, with the red line representing equal vote shares in both elections.
Parties to the right of the red line did worse in the recently concluded elections than in last year’s general elections, with the distance from the line indicating the difference in performance between the two elections.
The first takeaway from this graph is that both the BJP and Rashtriya Janata Dal (RJD) did much worse in the assembly elections than in last year’s Lok Sabha elections, though we should keep in mind the differences in seat contested.
Kumar’s Janata Dal (United), or JD(U), on the other hand, actually improved its vote share from last year, as did independents.
While this measure is flawed (as mentioned earlier), it summarizes the story of the elections well, and explains why the Grand Alliance got such a massive majority. For not only did the vote share of the NDA decrease compared with last year’s general elections (overall vote shares of all NDA parties fell), but its three major opponents came together to form the Grand Alliance.
The 38% vote share obtained by the NDA (putting together the BJP, Lok Janshakti Party, or LJP, and Rashtriya Lok Samta Party, or RLSP) in 2014 was enough for it to win three-fourths of the seats in a clear three-cornered contest.
But with the opposition coming together in the Grand Alliance, a combined 34% would never be enough to come even close to majority.
In order to make better sense of vote shares (given the difference in the number of seats contested), we will next look at the “contested vote share”, which ignores seats not contested by a party. In order to calculate this, we divide the total number of votes won by a party by the total number of votes cast in all constituencies the party contested in. Figure 2 compares the contested vote share from the 2014 general elections with the contested vote share in the assembly elections.
The first thing to note is that the BJP’s performance dropped only marginally compared with the 2014 general elections. Despite significant campaigning by the BJP leadership, there was little impact from it. The BJP’s alliance partners dropped their vote share precipitously, with the LJP and the RLSP both going down significantly in contested vote share.
While the BJP’s performance might have helped it maintain its share of seats, the coming together of the Grand Alliance thwarted such plans. The JD(U) had put in a pathetic performance in 2014, with a paltry 17% contested vote share.
With the coming together of the alliance, this increased to over 40% in this year’s assembly elections. The RJD also put in a spectacular performance, increasing its contested vote share from 30% to 44%.
When the JD(U) and the RJD came together, commentators were concerned that votes would not transfer easily from one partner to the other, given the history of rivalry between the two parties.
Based on the above data, it appears that any such leakage would have only been marginal.
Coming back to pollsters, there are many reasons why they might have got it wrong.
Firstly, it appears that they did not estimate the effectiveness of the Grand Alliance in a proper manner. The right way to have measured alliance effectiveness would have been to ask voters who they voted for in the last assembly elections in addition to who they were voting for in the current elections.
This would have allowed pollsters to calculate leakage of votes away from the Grand Alliance, if any. Given available data, it is not clear if the pollsters asked such questions (sadly, the lack of disclosure norms mean that questionnaires are not available).
Secondly, while the BJP managed to hold on to its vote share, pollsters seem to have underestimated the drop in the vote share of its allies. It is common to assume in opinion or exit polls that all partners of an alliance perform similarly. This assumption came unstuck in these elections, with BJP votes not “transferring” properly to its allies.
Finally, there was the issue of independent candidates, whose overall vote share shot up by over five percentage points. Independent candidates are hard to model in opinion or exit polls since independent candidates are independent of each other, and numbers from one constituency cannot be extrapolated to find out the numbers in an adjacent constituency. Some independents are party rebels, which requires special modelling (this part cannot be automated).
Given the difficulty in modelling, pollsters have a tendency to ignore, and sometimes underestimate, the effect of independents. With the vote share of independents shooting up in these elections, this tendency proved to be costly.
Overall, this was an unusual election and a hard one to call. Election Metrics has repeatedly pointed out that the lack of “prior models” (in the wake of redrawn alliances) would make polling hard.
The inconsistent performance of the NDA (with the BJP holding vote share but allies dropping) made things harder still for pollsters, as they overestimated the NDA’s vote share. With independents and “others” turning out to be much more significant in these elections compared with other elections, it would have taken either special skills or extraordinary luck to call this election correctly.
Tailpiece: Election Metrics had always maintained that this election was unlikely to be close and the odds of it being close had been grossly overestimated. We turned out to be right, though the data based on which we made the call turned out to be flawed—given that the Grand Alliance had a remarkable lead over the NDA in terms of vote share itself.