Sampling the truth on ground

Sampling the truth on ground
Comment E-mail Print Share
First Published: Tue, Jul 24 2007. 12 18 AM IST
Updated: Tue, Jul 24 2007. 12 18 AM IST
The number of HIV affected persons in India has come down from 5.7 to 2.47 million after the National Family Health Survey-3—a survey far more extensive than any carried out earlier. The difference is enormous and it highlights the perils of playing with numbers in the Indian context. Unfortunately, the reactions so far betray the ignorance with which the media and activists handle numbers.
Some have asked the rather obvious question: where have all these people gone? One simple answer is that they never existed. But, the truth is a little more complicated. Both the numbers are extrapolations, the second by direct sampling of small sets of the population, the first through indirect means. It means there is no way to compare one with the other. The question that should have been asked in each case, by every activist quoting the numbers and every journalist reporting it, is, how reliable are the figures, how much error is involved in the estimate? Without an understanding of how much an estimate may be off the mark, its usefulness in policy is more than questionable.
The reason to go with the current estimate is that it is more reliable, but only after we know the likely error involved. It, however, does reflect the unreliability of the indirect method earlier used and the perils of planning with bad data. It is as if in the absence of reliable numbers, any number sufficed as long as it would focus attention on the problem. The question about error margins needed to be asked then, it needs to be asked now.
Activists have reacted with righteous indignation to the new numbers, but that does not necessarily make them right. Advocacy often has exaggeration built into it and numbers can be dangerous tools. Some of their statements almost seem to imply they would have welcomed a doubling of the numbers rather than a halving. In reality, the fact that the prevalence rate may be lower should be reason to intervene more aggressively—intervention now has far better chances of containing what remains a significant health threat.
It should be clear, however, that the current numbers would certainly be off the mark. The simplest analogy is with the pre-, exit- and post-polls surveys carried out at the time of elections. Although each round does narrow down to the final outcome, poll predictions in India remain little better than astrology. The problem is the lack of reliability in extrapolating from a sample to the entire population. This is a problem far more marked in India where the diversity makes it difficult to predict across regions and jatis.
The problem of jatis is key to sampling methods in this country. The census data does not provide accurate information because this data is never collected, methodologies developed in the West do not take account of a complication they have no awareness of, and when you combine these two facts, sampling in India becomes a precarious business.
A similar debate has been played out with poverty estimates, based on the National Sample Surveys. As a result of a change in sampling methods in the late 1990s, figures now available are not comparable with pre-1995 figures. While many commentators boasted of the decrease in poverty figures from 36% to 26%, others argued the comparison was meaningless. In the end, the figures became an excuse, you could predict the arguments simply by whether the writer had a pro- or anti-reform position. New data is likely to emerge by year-end and we will see the same tiring playout again.
The debate needs to be recast. Exactly as is the case with 2.47 million HIV patients in this country, even the best estimate of one in four Indians below the poverty line is hardly cause for celebration. What remains true is that liberalization has enhanced the government’s capacity to do something about poverty. Perhaps the real debate should be whether wealth will trickle down or whether intervention needs to be far more aggressive. Again, questions about likely errors in the estimate are important.
In each of these cases, we will be best aided by good journalism from the ground. This provides a certain assessment not available through figures, activists and policymakers.
The few journalists who did an honest job of travelling through UP were far better at predicting the Mayawati victory than the surveys. The ones who toured Andhra before Chandrababu Naidu’s debacle were far more accurate about his fate than the claims about poverty reduction based on surveys and journalists who have followed HIV/AIDS have long expressed doubts about the reliability of the data.
While we wait for statisticians to improve their methodology— not impossible in a country where P.C. Mahalanobis devised the idea of sample surveys—and the country to become more honest about caste, newspapers need to get their senior staff to worry less about dining with ministers and bureaucrats and more about travelling across the country. Rather than parroting numbers quoted at some Delhi press event, they need to be able to bring something vital to the debate.
Hartosh Singh Bal is former chief of bureau at Tehelka. His interest in mathematics continues, even after a master’s degree in the subject from New York University. Comments are welcome at theirview@livemint.com
Comment E-mail Print Share
First Published: Tue, Jul 24 2007. 12 18 AM IST
More Topics: AIDS | HIV affected | Survey | Views | TheirView |