How India’s statistical system was crippled
20 min read . Updated: 07 May 2019, 11:41 PM IST
PremiumThe inside story of the decline of a globally renowned statistical system and its effect on Asia’s third largest economy
New Delhi: Toothless. Powerless. Impotent.
These are the adjectives used to describe India’s apex statistical organization, the National Statistical Commission (NSC). And these haven’t been hurled by its detractors but by three former members of the NSC who have been part of the commission at different points in time.
The NSC shot to prominence in December last year after two of its members resigned, citing undue government influence in the publication of a National Sample Survey Office (NSSO) report on jobs, which the NSC had cleared (that report is still under wraps although leaked portions of the report have been published in Business Standard). Today, the NSC is defunct with no new independent members appointed by the government (NSC needs five members to function).
The suppression of the jobs report that led to the resignations was only the last straw that broke the NSC’s back. A Mint investigation reveals that the systemic undermining of the NSC began soon after it was constituted in 2006, and has continued under successive governments at the centre since then. The NSC and the office of the chief statistician of India (CSI)—who apart from being the secretary to the ministry of statistics and programme Implementation (MoSPI) is also the secretary to the NSC— have been at loggerheads for most of this period, fighting battles over turf and resources.
A divided leadership meant that several reform proposals that could make the statistical system more robust, transparent, and autonomous failed to make headway. The story of the NSC’s emasculation is part of a broader story of the decline of what was once a globally renowned statistical system. It helps us understand why the credibility of India’s official statistics has hit rock-bottom in recent years.
Over the past few years, a growing number of economists and analysts have raised questions about India’s new gross domestic product (GDP) series. One of the key bones of the contention is related to the use of a new database on companies, MCA-21. Critics had pointed out that this database could include ghost firms that exist only on paper, and had demanded that the database be made public.
A “technical" report published by the NSSO shows that the fears of the critics were justified (Also Read: New GDP series faces fresh questions after NSSO discovers holes). When a survey based on the database was launched in 2016, it was found that many of the companies were not traceable, or had closed, or operated in a different line of business than what they had reported. The results of the survey were so disappointing that the two detailed reports that were being planned based on the survey had to be junked.
The use of the untested database in national accounts, the lack of transparency about it, and the lack of any effective audit of the GDP database all point to the lack of effective oversight of the Central Statistics Office (CSO), which is responsible for the National Accounts Statistics. The weakening oversight of the CSO is directly linked to the story of the NSC’s emasculation since it was first set up.
This story has been pieced together based on an analysis of thousands of pages of official documents—annual reports, minutes of meetings, and committee reports—and from oral testimonies of 16 people who worked in various capacities in the statistical system (including former NSC members, official statisticians who worked with the CSO and the NSSO, and academics who have been part of various advisory committees set up by these organizations over the past two decades). Ten of these interviews were conducted in person. The rest were conducted over phone. Several of those interviewed declined to be identified.
NSC, a halfway house
The NSC was initially set up through a government resolution in 2005 and constituted in 2006 with a part-time chairman (Suresh D. Tendulkar) and part-time members. Pronab Sen was appointed as the first CSI in early 2007 to function as a secretary to the NSC and as the secretary to MoSPI.
A draft National Statistical Commission Bill with provisions for full-time members was discussed and approved by the first NSC in early 2008 but it never saw the light of day. This was the first of two such failed attempts to empower the NSC with more teeth and resources, with a third attempt currently hanging in the balance.
The objections in both cases seem to have come from the CSI, who according to former NSC members, stalled efforts to empower the NSC. India’s first CSI, Pronab Sen, said he objected to the draft Bill, as he felt that a body working “outside the government" would never be able to get other departments to share data. The twin power structures in the new statistical architecture in fact left plenty of room for ambiguity and conflict, and contributed to the bad blood between the CSI and the NSC.
“The staff at the NSC secretariat is MoSPI staff and owes its allegiance to the secretary (CSI)," said Amitabh Kundu, economist and a member of the first NSC headed by Tendulkar. “The secretary controls everything, down to printing funds." “The NSC chairman has a minister-of-state rank but since the CSI is the MoSPI secretary, he can stall things citing procedural issues," said Kundu. “Even during the tenure of the first NSC, several decisions were taken by the CSI which should have come to the NSC. Instead, we were just informed. Nobody protested outside NSC meetings as that was not very common those days."
In March 2010, Sen modified the composition of the Standing Committee on Industrial Statistics (SCIS) without consulting the NSC, earning the NSC’s ire. “There has to be clarity on authority and accountability," said Sen. “The CSI is accountable to the government, the NSC is not." (The NSC is accountable to Parliament). Sen said he does not recollect the SCIS incident but added that he thought the NSC had no business to decide on the SCIS.
“The CSO partially modified the composition of the SCIS on 11th March 2010 without making a reference to the NSC and perhaps without the approval of the Minister of Statistics," the NSC annual report of 2010-11 noted. “The NSC expressed its strong displeasure on the modifications made by the CSO ... It was decided that the Committee would be constituted by the NSC."
By this time, R. Radhakrishna had become the chairman of the NSC, and he set up a committee headed by the legal luminary, N.R. Madhava Menon, to examine a law to provide teeth to the NSC, and to provide clarity about its roles and responsibilities. The Menon committee in its report submitted in 2011 suggested an empowered NSC, with members selected jointly by the government and the opposition, and answerable to Parliament rather than to the government of the day.
The Menon committee report and the draft bill it recommended did the rounds of various ministries, including the law ministry (which cleared it), but did not find favour with the government. “I think T.C.A. Anant (who had become the CSI by then) did not want the law, and kept stalling," a former NSC member said. Anant declined to comment.
“In my opinion, lack of active involvement by the then CSI for whatever reason, is the main cause for delay in implementation of Menon committee report," said M.V.S. Ranganadham, a former statistician who was member-secretary to the Menon committee, in an emailed response to questions. “The CSI is supposed to pilot the NSC Bill in the government circles and enlist political and bureaucratic will."
“The dual role assigned to the CSI, in my opinion, did not work well in the system," added Ranganadham, who retired as the director general of CSO last year. “As secretary, NSC, he is supposed to assist NSC in regulating the system. As secretary, MoSPI, he is the head of main statistical organ of the system, which has to be regulated. Why only CSI? For that matter, nobody in the system, given a choice, would like to be regulated."
The lack of statutory backing and the struggle for identity meant that NSC members often felt their recommendations were falling on deaf ears. “As far as NSSO surveys are concerned, the NSC has regulatory powers, since it took over the role of the governing council," said a former NSC member. “But its oversight of CSO’s activities is very limited. CSO could ignore it when it wanted, and did so when it suited them."
This lay at the heart of the CSI-NSC conflict during the term of the last NSC. But to understand this conflict and why the GDP estimation process remains mired in controversy, it is useful to look back at the first credibility crisis the statistical system faced, and the conditions under which the NSC came up.
The first credibility crisis
The first big crisis arose when the 1999-2000 consumption expenditure round of the NSSO showed a sharp spike in average consumption expenditure, which in turn translated into a sharp decline in poverty in the 1990s. But the 1999-2000 survey was based on a different recall period. Critics argued that the change in the recall period rendered the new estimates incomparable with the past.
Several scholars including Angus Deaton re-examined the raw data to analyse the trends in those components of consumption expenditure that were less likely to have been affected by the methodological changes in the survey, and suggested that while there was indeed a fall in poverty levels, the fall was likely to have been less than that suggested by the official estimates.
This story begins in the late 1990s, when the NSSO conducted a series of experiments which showed that different reporting periods of consumption—i.e., whether households were asked to recall consumption over the past seven days or the past 30 days—led to different averages of household expenditure. Historically, the NSSO had used a uniform 30-day recall period. When that was changed to a mixed approach—with shorter recall window for high frequency items, and longer recall window for low frequency items—the results seemed to suggest a bump in consumption, and less poverty.
But it was not clear which approach was the correct one. A working group was set up to answer this question, and its report published in 2003 seemed to suggest that the old method was adequate. However, the report of the working group was not available when the 1999-2000 survey was conducted, and this led to a rushed adoption of the new methodology.
Deaton and other scholars suggested that the rushed adoption was perhaps driven by the harsh criticisms levelled at the NSSO by a section of economists, who felt that the poverty decline reported in earlier surveys were too low to be true given the high growth rate India was experiencing.
What they say about jobs now, they said about poverty then.
Among other things, the episode highlighted the need for insulating official statistics from the pulls and pressures of politics. As luck would have it, around the same time, a high-level committee led by the former Reserve Bank of India (RBI) governor, C. Rangarajan, presented a plan to do exactly that.
The Rangarajan Commission, set up to suggest ways to overhaul the Indian statistical system, suggested the creation of a permanent and apex statutory body, the NSC that would be “independent of the government and responsible to the Parliament in respect of policymaking, coordination and certification of quality of core statistics".
The NSC-CSI tensions were at their lowest ebb when the former CSI, Sen became the NSC chairman. Sen says there was no conflict with MoSPI when he was at the NSC. The NSC-CSI tensions escalated after Sen’s term as NSC chairman ended and Radha Binod Barman took charge in 2016. Former NSC members said that apart from starving it of funds, the NSC was not consulted on several issues that were under its purview.
The second credibility crisis
A key area of disagreement between the last NSC and the CSI pertained to the new GDP series. The new GDP series (base year 2011-12) ran into controversy soon after it was released in 2015. Critics pointed out that the series failed basic “smell tests" and was at variance with what other macroeconomic indicators were telling us about the economy.
A key area of contention was the manner in which corporate sector growth had been estimated. The estimation of corporate sector growth in India has been a major weakness for quite some time. A statistical system designed to measure economic activity in a state-led economy struggled to keep track of economic activity as the share of the private sector grew.
Historically, the CSO relied on a sample survey conducted by the RBI to compute estimates for the corporate sector. The RBI sample consisted of only a few thousand companies, and hence the estimates were blown up (or multiplied) in proportion to the coverage of the paid up capital of the sample companies to the total number of companies registered with the ministry of company affairs (MCA).
This methodology was questioned, among others, by the Rangarajan Commission, which pointed out that the presence of a large number of fictitious or shell companies in the MCA records tended to lend an over-estimation bias in the GDP numbers.
Ahead of the base-year revision in 2015, a sub-committee appointed by the Advisory Committee on National Accounts (ACNAS) suggested the use of a new database, MCA-21, to construct the new GDP series. But instead of using the estimates generated from the database directly, as agreed upon by the sub-committee, the CSO scaled up even these estimates to account for non-reporting companies, which had declared returns in earlier years. The CSO said it was done at the eleventh hour, and in consultation with the chairman of the sub-committee, B.N. Goldar, a former professor at the Delhi-based Institute of Economic Growth.
However, not all sub-committee members were consulted, or even informed, about this change. One sub-committee member, R. Nagaraj, economist and professor at the Indira Gandhi Institute of Development Research, Mumbai, learned about it only after the release of the new series, and pointed out that the “discredited" scaling up methodology had led to an over-estimation bias once again.
The root of the problem was that no one really knew, or bothered to ascertain, the true size of the universe of genuine companies in India. Nor was any attempt made to verify the extent to which MCA-21 data was accurate, or to validate it using other databases (such as the Annual Survey of Industries, or ASI). As it turns out, NSSO field staff has now found that many companies reported to be “active" companies by the MCA are actually ghost companies that exist only on paper.
The CSO replied to Nagaraj’s criticisms in an Economic and Political Weekly paper that year but it failed to convince many economists. Among them was Rajiv Kumar, currently the vice-chairman of NITI Aayog, who was then working at the Delhi-based think tank, Centre for Policy Research (CPR). In a 2015 CPR policy brief (“The Curious Case of GDP Growth"), Kumar expressed scepticism about the CSO’s response, and suggested that “the final estimates might be considerably overestimated".
“The government, which has come into office one year ago and under whose watch these revisions have been announced, must ensure that the credibility of its statistics is above all doubt and provides a real and honest basis for policy making," wrote Kumar. “For this, it must require the CSO to not only review the methodology but make if far more transparent by further detailing it in the public domain."
Kumar did not respond to an email asking whether he followed up on this issue after joining the government.
There were other questions raised about the new methodology, including the use of formal sector indicators to estimate informal sector growth, such as using growth rates of the organized manufacturing industry to estimate growth for unorganized manufacturing. Others pointed to the potential mis-classification of industries, which meant that growth in one sector might well have been attributed to another. Still others argued that the use of inappropriate deflators tended to overstate the real GDP growth numbers, i.e., the GDP growth adjusted for inflation.
In a 2018 research paper, the Ahmedabad-based economist and member of the RBI’s monetary policy committee, Ravindra H. Dholakia, pointed out that in the early national accounts figures, state-level data was not available, and the “allocation" (or proportion) of all-India estimates were used to derive state-level estimates. But the proportion of such allocated estimates had been declining with each successive base-year revision till the 2004-05 revision, when the share of allocated gross state domestic product (GSDP) for Gujarat fell to about 30%. This meant that 70% of Gujarat’s GSDP was being measured through indicators that were directly linked to the state economy rather than through an indirect and ad-hoc “allocation" based on national figures.
However, this scenario changed completely with the latest base year change. The share of the “allocated" GSDP for Gujarat shot up to 74%, partly because of the use of the MCA-21 database, which did not provide state-wise breakup of the value added by companies. This meant that a large share of the Gujarat’s GSDP was just a rough approximation based on national figures rather than a real measure of the state’s economy. Other states too faced similar issues, though the magnitude of the problem varies across states.
Dholakia’s conclusion was simple and stark: the new series must be abandoned for state-level GDP estimation, and the old series continued till a further revision of the base year takes place. “We cannot throw the MCA-21 database out, but have to use it intelligently," said Dholakia. “Currently, its use has not been thought through carefully, particularly for estimation of state income."
The growing criticisms about the new GDP series prompted discussions at the NSC, and it was decided that the CSO should be asked to respond to the public criticisms about the GDP numbers. The CSO did present a reply but it was merely a restatement of its methodological note rather than a response. This prompted the NSC’s ire, in a meeting held at Mumbai on 30 July 2018. A copy of the minutes of that meeting held a few months before the NSC became defunct was reviewed by Mint.
Meanwhile, a sub-committee on industrial statistics reporting to an NSC committee on real sector, and headed by Goldar, pointed out several drawbacks and “inconsistencies" in the MCA-21 database, and suggested that the CSO should conduct studies and verify the data to improve the reliability of the database.
Goldar, who had cleared the untested database earlier, did not respond to an email asking why such verification was not done before the new GDP series numbers were finalized in 2014-15.
The long road to credibility
India wasn’t always like this, statistically speaking.
When the NSSO was first set up by P.C. Mahalanobis, initially as part of the Indian Statistical Institute (ISI), and later as an independent organization, it was the first organization to conduct nationally representative household surveys anywhere in the world.
Although remembered today as the architect of India’s five-year plan model, Mahanalobis, as the honorary statistical adviser to the cabinet, had a greater contribution in building a new statistical architecture for the country.
With unstinting support from India’s first prime minister, Jawaharlal Nehru, Mahalanobis helped establish the CSO, the National Sample Survey (NSS), and the ASI, all of which were run from ISI in the early years. In his 1998 book, The Idea of India, political scientist Sunil Khilnani argued that in courting intellectuals such as Mahalanobis, Nehru’s aim was to “subordinate the civil servants to the superior rationality of scientists and economists".
Mahalanobis still faced scepticism since surveys at that time were a new and exotic phenomenon, according to his biographer Ashok Rudra. But given the paucity and unreliability of administrative data, Mahalanobis impressed upon the Nehru cabinet the importance of having regular surveys to have reliable data on which to develop plans for the economy.
To establish the credibility of these surveys, he invited some of the global pioneers of statistics to review the work done at the ISI. Almost half a century later, Deaton would write: “Where Mahalanobis and India led, the rest of the world has followed."
The Rangarajan Commission had envisaged the NSC to collectively play the role that Mahalanobis played in Nehru’s government. But India had changed since Nehru’s time. Now, the rationality of scientists and economists was being subordinated to the will of bureaucrats.
Reflecting on the NSSO consumption round fiasco of 1999-2000, Deaton had argued that while politicians may not have directly intervened in the process, politics, in a broader sense, played a role since statisticians felt they should rush to adopt methodological changes, which even if statistically questionable, would lead to politically acceptable results.
Something similar might have happened with the GDP estimation process as well.
“Our official statisticians know that the costs of under-estimating GDP growth are far higher than that of over-estimating," said one of the former NSC members cited above. “If the numbers are being jacked up, either because of depressed deflators, or any other reason, you are less likely to be questioned."
“I think it is widely recognized that the national accounts in India are relatively weak," said Deaton in an interview to Hindustan Times after he won the Nobel Prize in 2015. “So what I am most worried about is that growth is not as high as the accounts show....Revisions that increase growth are more readily accepted than revisions that reduce growth. So I am more worried about growth being overstated than poverty being understated."
The debate could have been depoliticized if the CSO was more sensitive to criticisms, and had made proactive disclosures on the error estimates of different sub-sectors of GDP, with explanations for why output estimates for some sectors were more reliable than that of others. In fact, the first national account estimates presented by Mahalanobis after India’s independence carefully noted the data gaps and limitations of the estimates, as well as the error margins associated with each sectoral estimate. Providing such error estimates would also have drawn wider attention to data gaps, and could have helped MoSPI garner the requisite resources to fill those gaps.
The poverty debate also highlighted how changes made without adequate debate could turn out to be flawed and controversial: a lesson that seems to have been forgotten during the base-change exercise in 2014-15.
It showed the importance of transparency and the importance of releasing raw data, so that the debate is settled by researchers working on the raw data rather than by politicians. That lesson too seems to have been forgotten. The MCA-21 database was not made public despite repeated demands.
To promote transparency, the NSC had recommended as far back as in 2009 that the minutes of NSC meetings be made available online but that recommendation too was ignored. An audit by independent experts commissioned either by the NSC or a parliamentary standing committee could also have helped restore credibility while showing the path towards improving the national account statistics. That opportunity too was missed.
In conclusion
The road to improving credibility of official statistics will be a long one and there are differing opinions on how best to do it. But there is a broad consensus that much will depend on effective leadership.
“Resources used to be one big constraint earlier but today that is not the case," said one former NSC member. “Politicians and bureaucrats both realize the importance of data. Many ministries are already conducting surveys of their own and spending a lot on collecting data. A well-run and responsive statistical system can actually save costs for the government but there needs to be an effective leadership that can sell that idea to the political class and still retain independence for the statistical system."
There is disagreement regarding how far a law backing the NSC would help. But there is agreement that a restructuring of the NSC, with greater resources, and a well-defined role would help. A draft statistical policy was released for public comments by MoSPI in 2018 that envisaged a larger role for the NSC, incorporating some suggestions of the Menon committee. However, that policy has been scrapped and a revised policy is being prepared, according to two people familiar with the development.
“The Government will be finalizing the policy after due discussions and consultations in accordance with the established process," was all that a MoSPI spokesperson said, in an emailed response to questions about the draft policy.
After the recent controversies, policymakers have realized the importance of the NSC, Kundu believes. “There is now a realization that NSC can’t be under MoSPI," said Kundu. “Whichever government comes to power after the elections must empower the NSC."
