Statistical reform needs key stakeholder consensus for implementation

“As the system has been kept highly opaque, it has created a false sense of expertise… such as experts in national accounts or experts in consumer expenditure surveys… creating an impression that they (the deemed experts) are indispensable," the mail said. “(A) complacent attitude has made ISS officers averse to any change."

It is easy to dismiss the gentleman as a disgruntled officer who did not get the positions he fancied. But I have heard such complaints from scores of people associated with the statistical system over the past few years. 

Senior ISS officers have little training in handling big data, and lack knowledge of modern database management, a National Statistical Commission (NSC) member complained to me. Another statistician pointed to the inability of the statistics ministry to set up a decent data warehouse as evidence of its decaying capabilities.

Also read: The spatial spread of economic activities must be mapped for data accuracy

The declining autonomy of the statistical system and its weakening capabilities may be intertwined in a vicious cycle today. A weakened statistical system is unable to justify the estimates it generates, or explain contradictions between different data-sets. 

This in turn allows politicians and politically motivated technocrats to meddle in statistical affairs. Officers who can ‘manage’ political demands end up climbing the career ladder, demoralizing others.

The absence of incentives for statistical innovation compounds problems. Statistics, after all, is an applied science, and progresses through experiments. Ideally, the National Statistical Office (NSO) should have a dedicated unit for conducting pilot trials and methodological innovations. 

Once innovations are tried and tested in a statistical sandbox, data users should be made aware of what the innovations entail, and only then should changes be rolled out. What we currently have is a system of ad hoc changes. Even when small-scale trials are conducted before effecting such changes, the trial data is almost always withheld.

India’s statistical crisis is now too severe to be ignored. As reported by this newspaper earlier this year (‘India’s statistics system is a mess. The PMO is looking in’, 23 January), the Economic Advisory Council to the Prime Minister (EAC-PM) has prepared a policy paper on the sorry state of the statistical system, with suggestions for reforms. 

Also read: PMO review: Statistical clarity is a must for successful governance

Some reform suggestions have also come from individual EAC-PM members, according to two people familiar with these developments. The need to revamp India’s statistical infrastructure found mention in the ruling Bharatiya Janata Party’s manifesto for the 2024 Lok Sabha elections. 

A person who attended a meeting on this subject at the Prime Minister’s Office (PMO) in March said that there is considerable agreement on the need to upgrade the statistical system. But the path towards that upgrade still remains hazy.

The new government will be doing itself and the country a favour if it sets up a high-level task force to conduct a detailed assessment of the state of the statistical system and outline a time bound path for reforms. As I have argued earlier (‘India’s Statistical System: Past, Present, Future’, Carnegie Endowment WP 2023), this committee should first of all conduct a cost-benefit analysis of the existing public data pipeline. 

It should examine how much key ministries spend on ad hoc surveys and whether the government and the public will be better served if fewer high-quality surveys are conducted under the supervision of a statistical authority. It should also provide indicative assessments of the costs involved in keeping data in silos, and the benefits that might accrue when economic statistics of different kinds are integrated.

Secondly, the task force should suggest legal and organizational changes that can help create an effective and autonomous statistical system. Official statistical work must be subjected to review by technical experts or statistical auditors, who should be answerable to Parliament, not the government of the day. The National Statistical Commission was set up to play such a role. But the lack of statutory backing for the Commission ensured that it remained a toothless regulator.

Finally, the task force should prepare a medium-term statistical strategy document. This document should set out guidelines on documentation and statistical communication. It should indicate steps to foster greater innovation. For instance, it could outline human resource policy changes that encourage statisticians to do better research.

It will be possible to improve statistical processes and products on a sustained basis only if all stakeholders share a broad consensus on what ails the statistical system, and what must be done to fix it. Without such a consensus, the reform drive will falter.

Also read: Need a credible plan to fix India’s statistical system

It is worth noting that there was a major thrust to revamp the statistical system five years ago. However, that modernization drive ran out of steam because of a lack of buy-in from key stakeholders. Hopefully the government will learn from that experience, and get all stakeholders on board this time.

A credible statistical system can help us find agreement on basic economic facts, and improve the quality of public discourse in the country. It will also help policymakers and businesses avoid costly mistakes.