In a recent poll 47% of around 1,000 respondents said they will vote yes in the forthcoming referendum, but what is the actual probability of a yes vote?
Much has been made of a recent poll by YouGov in which 47% of around 1,000 respondents said they will vote yes in the forthcoming Scottish independence referendum. This is significantly higher than the 43% who said yes in the last poll held in mid-August and has resulted in analysts believing that it is now indeed plausible that Scotland might break away from the United Kingdom. The question, however, is if 47% of the sample said yes, what is the actual probability of a yes vote?
Unlike Indian elections, which are rather hard to forecast thanks to multi-cornered contests and the parliamentary system, binary contests such as the Scottish referendum are rather easy to analyse.
The proportion of people who say yes is driven by a binomial distribution, and when the sample size is large (as it usually is for opinion polls), a simple formula allows us to estimate the probability of a true yes vote as a function of the number of people surveyed and the number of people who say yes, as long as the sample is random.
While the math is not too complicated, for the purposes of this article, it suffices to know that the larger the sample size (number of people surveyed), the more confidence we can have in our predictions. For example, if we interview 100 people and 53 of them say yes, we can infer based on the poll that there is a 73% chance that a majority of the population say yes.
If we have surveyed a 1,000 people, however, and if 530 say yes (just to keep the proportion constant), then the probability of a majority of the population saying yes rises to a whopping 97%. The interesting thing to note here is that this calculation depends solely on the number of people being surveyed—the total size of the population doesn’t matter (this presentation by Rajeeva Karandikar has a more rigorous take on this result.
Coming back to the question, given that 47% of the respondents in the latest YouGov poll have said yes, what is the probability that a majority of the Scottish population will vote yes in the referendum? YouGov polled 1,063 people, of which 961 gave a definite response (the remaining said they were undecided). Of the 961, 450 (or 47%) said they will vote yes in the referendum. Plugging it into our formula, this indicates that the probability that the majority of the voting population will vote yes is 2.4%. Yes, you read that right—the latest survey indicates that the possibility of a yes vote is a rather paltry 2.4%.
The question that arises is why analysts are getting excited about this 2.4% possibility, given that it is still a rather small number.
For this, we need to go back to the earlier polls. In the immediately preceding opinion poll, held in mid-August, 43% of the respondents said they would vote yes. For a survey size of 1,000 respondents, that would give the probability of the majority of voters saying yes at 0.0004%. In comparison to that, a 2.4% chance of a yes vote is massive.
It is also instructive to look at these polls using the “possible, plausible and probable" framework used by investor Bill Gurley and valuation guru Aswath Damodaran in their riveting back-and-forth on Uber’s valuation in July.
Until the previous round of polls, a yes vote in the referendum was possible, but definitely not plausible.
The latest round of polls, however, pegging the chance of a yes vote at 2.4%, makes a majority yes vote not only “possible", but also “plausible".
At current levels, though, we can say that a yes vote outcome is definitely not “probable". Thus, the latest polls are significant in terms of the plausibility of a yes vote, which is a major change from earlier polls. Thus, analysts have reason to be excited.