Home / Opinion / Columns /  The greatest Indian statistical debate

When we think of India’s big data debates, two immediately spring to mind. The first and most recent is the debate on India’s gross domestic product (GDP) series introduced in 2014-15. The second is the debate on India’s poverty decline since liberalization, triggered by methodological changes in the National Sample Survey (NSS) consumer expenditure round of 1999-2000.

Yet, it is a debate from a more distant past that could arguably be called the greatest statistical debate of all time in the country. This debate, between the key proponent of NSS, Prasanta Chandra Mahalanobis, and NSS co-founder, Dhananjay Ramchandra Gadgil, occurred at the founding moment of the NSS in 1950-51.

It was a spirited battle between the two intellectual giants and the respective institutes they founded: the Indian Statistical Institute (ISI) in Calcutta and Gokhale Institute of Political Economy in Poona. The concerns raised during the ‘Poona-Calcutta battle’ of 1950-51 remain salient even today.

Most of us think of the NSS as ISI’s legacy. But it began life as a collaborative experiment between ISI statisticians and Gokhale Institute economists. Both organizations had conducted pioneering surveys in their respective regions during the British Raj. The ISI team had conducted the first set of scientifically designed crop surveys in Bengal. The Gokhale team did surveys on rural economic issues in the Bombay province.

After India’s independence, Mahalanobis became the statistical advisor to the Union cabinet, and asked for Gadgil’s help in rolling out the first cross-country household survey. The aim was to establish a national survey that could meet the data needs of different ministries, and help fill data gaps identified by the National Income Committee (chaired by Mahalanobis with Gadgil and V.K.R.V. Rao as members).

Very soon it became apparent that there were major differences of views. The Gokhale Institute team led by Gadgil and V.M. Dandekar felt that the objective of the ‘multi-purpose survey’ was not well defined, and that the questionnaires were likely to become cumbersome. They also had grave concerns about the proposed survey design.

A compromise was reached at the eleventh hour since the disagreements couldn’t be resolved. The ISI team would use the ‘Calcutta schedules’ and their survey techniques to interview one sub-sample. The ‘Poona schedules’ and the alternate survey approach from the Gokhale institute would be used for the other sub-sample. The tests of both approaches would be settled in the field.

Fate had other plans. Gadgil felt that the early results sent by enumerators were deeply unsatisfactory. He advocated an end to the NSS experiment. A poor country such as India could not afford such an extravagance, he argued; instead, the funds for the project should be spent on strengthening the existing administrative data structure and state statistical bureaus. Gadgil also lent support to fellow Punekar P.V.Sukhatme, who wanted the agricultural machinery rather than the NSS to estimate crop yields.

Gadgil, Dandekar and Mahalanobis exchanged letters to clarify their respective positions and then met to resolve their differences. But these persisted. Mahalanobis agreed with some of Gadgil’s criticisms. But he argued that any new initiative faces teething troubles, and felt that the NSS data quality would improve over time as enumerators gained experience. He felt that existing administrative data were often biased because administrators doctored the figures. Hence, an independent survey agency was essential.

Mahalanobis’ influence on the government of the day meant that he had the last word. NSS was scaled up quickly, and the rest, as they say, is history. It became a global template for conducting household surveys to fill data gaps. In the early years of the republic, the NSS rounds offered a vast range of data to Indian policymakers.

The NSS is still an invaluable resource for economic policymaking given India’s large informal sector. But its use to fill data gaps has meant that the weaknesses in India’s administrative data structure never received adequate attention, as Gadgil had feared. Half a century later, another statistician from Maharashtra, S.M.Vidwans, would echo Gadgil’s critique while lamenting the decay in India’s administrative data systems.

Unlike the politicized attacks that NSS would face later, Gadgil’s criticisms were motivated by public interest. Gadgil’s concerns about the lengthy NSS questionnaire and the long recall period (of one year) are today accepted as valid concerns even by India’s statistical establishment.

The famous dissent of the Bombay school of economics (led by C.N.Vakil and P.R. Brahmananda) that opposed the Nehru-Mahalanobis plan for state-led industrialization is widely remembered today. The forgotten dissent of the Pune group of statisticians and economists holds equally important lessons.

People recall Gadgil today as the author of the famous ‘Gadgil formula’ that removed discretionary powers of the Planning Commission in allocating funds to states. He is also fondly remembered as an institution builder, and as a leader of the rural cooperative movement. It is Gadgil’s statistical legacy—as a pioneer of household surveys and as the dissenting co-founder of NSS—that deserves more attention.

This is the fourth of a five-part series on the founders of India’s statistical system.

Pramit Bhattacharya is a Chennai-based journalist. His Twitter handle is pramit_b

Catch all the Business News, Market News, Breaking News Events and Latest News Updates on Live Mint. Download The Mint News App to get Daily Market Updates.
More Less
Subscribe to Mint Newsletters
* Enter a valid email
* Thank you for subscribing to our newsletter.

Recommended For You

Get alerts on WhatsApp
Set Preferences My ReadsWatchlistFeedbackRedeem a Gift CardLogout