Narratives of statistical decline require much corrective rework

To start with, it is worth pointing out that this paper has bought into a Narrative of Decline. See, for example, Social Science Research Capacity in South Asia: A Report (2002) by Partha Chatterjee et al, and more generally, Cultural Pessimism: Narratives of Decline in the Postmodern World (2001) by Oliver Bennett. The way this Narrative is structured for India is that there was a Golden Age of Official Statistics in the 1950s and 60s, followed by a period of decline for almost 30 years. This was then followed by what can be best described as a stuttering revival, which in recent years has stalled with signs of decay. The Narrative is spun with some examples. The trouble with this analysis is that it is factually wrong.

The problem with this assessment of a Golden Age in the 50s and 60s (and the implicit subsequent decline) is that India had stood out for two reasons: first, the low base of the colonial system on which the post-independence apparatus was being built; and second, the poor shape of official systems in most of the developing world and the relatively limited data even from the developed world. This made India’s efforts seem quite remarkable. So, while India pioneered sample surveys, the early surveys were conducted by a fragmented system, with responsibility shared between the Indian Statistical Institute (ISI) and the National Sample Survey (NSS) directorate. The shared responsibility and frequent changes in design led to extensive delays and problems in implementation. A Review Committee comprising of the then-Cabinet Secretary B. Sivaraman, Professor V.M. Dandekar and R. R. Bahadur was set up in 1970 to review the NSS. The consequence of this was to create the current structure of the National Sample Survey Organization (NSSO) with in-house capacities for design, data collection and data processing. The system of quinquennial surveys for employment and consumption is also a consequence of this Committee. The standardization of definitions and design created a set of comparable surveys which have been appreciated by many users. Instead of the posited ‘decline’, the 70s and 80s actually mark a period when sample surveys truly came into their own. One criticism which remained of this period was of opaque and iniquitous access to disaggregated survey data, which was rectified in subsequent decades. Today, India follows a very open standard for dissemination of sample survey data.

A second example of the improvements in official statistics that occurred during the 70s and 80s relates to the computation of State Domestic Product estimates. The Fourth and Fifth Finance Commissions highlighted problems they faced in using state income estimates. Partly in response to this, the Central Statistical Office (CSO) set up in 1972 a Committee on Regional Accounts under the chairmanship of Professor Moni Mukherjee to standardize the production of State Domestic Product estimates. The work of the CSO and state governments in subsequent decades was in response to that panel’s recommendations. The recent committee under Professor Dholakia continues the same effort. An assessment of those recommendations, however, requires a separate column.

Having noted the flawed narrative of decline, I now turn to a second problem in assessing official statistics. Improvements in data collection, analysis and dissemination have been continuous. But improvements in data quality create problems for data users, especially those who do temporal comparisons. When earlier narratives built on poorer quality data are challenged by better data, many find it easier to question the data than correct the Narrative. A famous example of this towards end of the 19th century was an effort in physics to protect the fabled existence of Luminiferous Ether and question the Michaelson-Morley experiments. Thus we see that data users though constantly demanding improvements in data, can be simultaneously reluctant to accept the same when they challenge conventional narratives. The recent use of ministry of company affairs data by the CSO is an example of this phenomenon.

Finally, I turn to a more complex issue: Suggestions to reform the statistical system. We must recognize that under the Constitutional schema, statistics is in the concurrent list. This means that for a variety of statistical work, state level and central agencies are equal partners. Many of the suggestions made seem to require a dominant Central agency enforcing reforms. This has been and is being resisted by many state governments. A second dimension is that administrative statistics are intimately linked to structures of administrative governance. Improvements in data are often closely linked to improvements in governance standards. As I noted in my previous column, there have been huge improvements in this sphere, but resistance to adopting these databases for use in official statistics needs to be addressed. Further improvements in these databases also requires appreciation of the challenges of governance reform. To illustrate, though Universal Civil Registration of births and deaths has been often cited as important statistical reform, achieving this is not about fiat, but about evolving policies to increase institutionalized access to medical care.

A final issue (which I will elaborate upon in a later column) is that while the use of statistical techniques has become easier with modern technology, our understanding of statistics and the quality of statistics education have not kept pace.

T.C.A. Anant is a former chief statistician of India