Active Stocks
Thu Mar 28 2024 15:48:07
  1. Tata Steel share price
  2. 155.90 2.00%
  1. ICICI Bank share price
  2. 1,095.75 1.08%
  1. HDFC Bank share price
  2. 1,448.20 0.52%
  1. ITC share price
  2. 428.55 0.13%
  1. Power Grid Corporation Of India share price
  2. 277.05 2.21%
Business News/ Politics / Policy/  What cross-tabulating can tell us
BackBack

What cross-tabulating can tell us

Reports of opinion polls seldom include insights drawn by correlating responses to two or more survey questions

A file photo of Prime Minister Narendra Modi. Photo: Hindustan TimesPremium
A file photo of Prime Minister Narendra Modi. Photo: Hindustan Times

This might sound like a rather sweeping and uncharitable statement to make, but it seems political surveyors and reporters of political surveys in India don’t do cross-tabbing.

Reports of many political opinion polls, for example, report the overall proportion of responses to a number of questions, but they seldom include insights that can be drawn by correlating responses to two or more questions in the surveys.

Cross-tabulating might sound like a fancy term but it is rather simple. In the simplest case, you take responses to two questions in a survey, and count the number of responses to each combination of questions, and then record them in a two-dimensional table. That way, we can analyse the relationship between responses to various questions in the survey.

The Monday edition of this paper published the results of a survey conducted by InstaVaani on Prime Minister Narendra Modi’s approval ratings (mintne.ws/1KXlm7n).

The report said 74% of respondents “approved" of Modi’s performance, and went on to mention that 73% had voted for Modi’s Bharatiya Janata Party (BJP) in the 2014 general election (metros and non-metros combined).

Looking at these numbers, the first question that springs to mind is how these two are correlated, i.e., how many people who approve of the prime minister had voted for the BJP last year? And this is what a cross-tabulation (known in short as a cross-tab) tells us.

We simply count the number of responses for each combination of questions and note it in a table, which gives us the cross-tab (this can be generated in MS Excel using the Pivot Table function).

For the top left cell in table 1, for example, we count the number of respondents who mentioned that they had voted for the BJP in the 2014 general election AND mentioned that they approve of the prime minister’s performance.

The middle cell in the top row says that 426 respondents approve of the prime minister, but had voted for the Congress party in 2014. And so forth. The total doesn’t add up to the total number of people surveyed, since not all respondents answered the question on who they had voted for.

It is easy to notice that there is already much richer information about the survey than what was presented in the one-dimensional graphs in Monday’s paper. While it might have been tempting to equate the prime minister’s 74% approval rating to the 73% of the respondents who had voted for the BJP in 2014, we notice that there is actually a significant crossover.

For example, the above table tells us that 12% (663/(663+4,729)) of respondents who had voted for the BJP in 2014 do not approve of the prime minister’s performance. Moving over one column, we see that more than half the respondents who didn’t vote for the BJP in 2014 approve of Modi’s performance, suggesting that the prime minister’s approval is rather robust.

Cross-tabulations need not be restricted to two dimensions. We can include another variable, for example, the party that the respondent would vote for if elections were held on the day of the survey. With three dimensions, we can’t use a simple table as above; hence, we will use a combination of two tables:

There are some intriguing patterns in the second two tables. For example, table 2a tells us that there is a significant number (approximately 700) of respondents who voted for the BJP in 2014, approve of Modi’s performance and yet will not vote for the BJP if elections are held today.

Table 2b tells us of the existence of almost a hundred respondents who did not vote for the BJP in 2014, do not approve of Modi’s performance, and yet will vote for the BJP if elections are held today.

And there are many more such insights to be gleaned if one were to examine the above tables further, or to look at cross-tabs of other variables included in the survey (approval of the prime minister’s performance on various issues, for example).

Given that there are so many insights to be gleaned from cross-tabulating, the question arises as to why it is not a very common practice in Indian surveying and reporting.

Firstly, it is a bit complex to understand, with all the two-dimensional and higher-dimensional tables, though that can be solved with some good explanations. Understanding cross-tabulation depends on conditional probabilities (what proportion of people approve of the prime minister GIVEN that they had voted for the BJP last year, for example), a topic that is notoriously hard to master in school.

Then, cross-tabulations can also point out some inconsistencies in the survey. For example, it doesn’t make logical sense that someone who did not vote for the BJP in 2014, and who does not approve of the prime minister will vote for the BJP if elections are held now. Cross-tabulation might unearth more such contradictions, which can cast doubts on the credibility of the survey.

Cross-tabulation can also make biases in the survey more apparent. This survey clearly suffers from selection bias since 73% of the respondents self-identified themselves as BJP voters in 2014 (this might be an overstatement given bandwagon effects, see mintne.ws/1hxJaUn), while the BJP got far less than that in the 2014 election (the party got 31%, but then this survey is restricted to urban areas, with almost half the respondents being from metros; so, the two numbers are not strictly comparable). The presence of the above cross-tabulations can reinforce the fact that this survey is biased.

So, given that the survey is biased, is there a way we can calculate the prime minister’s true rating? It is not hard to do. All we need to do is to re-weight the samples. Seventy-three per cent of the respondents of the survey said they voted for the BJP in 2014, and 88% of them approved of the prime minister’s performance. Of the remaining 27% (who didn’t vote for the BJP in 2014), 61.5% approved of the prime minister’s performance.

Considering that actually only 31% of all voters had voted for the BJP in 2014, the prime minister’s true rating can be calculated as 31% (voted for the BJP in 2014) X 88% (approval among BJP voters) + 69% (did not vote for the BJP in 2014) X 61.5% (approval among non-BJP voters) which gives us about 70% which is not far from the simple average of 74%. So, while the sample is indeed biased, the people’s stamp of approval for the prime minister is not in doubt.

Finally, note that in order to create cross-tabulations, we need the raw data (one line for each respondent). Indian surveyors are notorious for not making their survey data public (even after anonymising); so, such analysis may not be possible for future surveys, unless the surveyors themselves publish results in this format.

Unlock a world of Benefits! From insightful newsletters to real-time stock tracking, breaking news and a personalized newsfeed – it's all here, just a click away! Login Now!

Catch all the Politics News and Updates on Live Mint. Download The Mint News App to get Daily Market Updates & Live Business News.
More Less
Published: 12 May 2015, 12:01 AM IST
Next Story footLogo
Recommended For You
Switch to the Mint app for fast and personalized news - Get App