What the debate on NSS population counts missed out

One of independent India’s early achievements was to set up a large-scale household survey system. The National Sample Survey (NSS) set up in 1950 and the Sample Registration System initiated in the 1960s inspired similar experiments all over the world. Since the late 1970s, innovation in the official statistical system has slowed down even as the structure of the Indian economy began changing rapidly. The 1990s brought budget cuts and ill-considered administrative changes, creating a massive manpower crisis.

Political pressures on the NSS mounted even as it struggled to deliver the new demands being made on it. The Rangarajan Commission report of 2001 made several recommendations to help it regain its edge and to insulate it from political interference. Successive governments have implemented those recommendations in a half-hearted manner. The National Statistical Commission (NSC) was set up in the mid-2000s to regulate statistical activities, but it has not been provided statutory backing till now.

In spite of its limited powers, the NSC has been able to intervene on some critical issues. One such issue pertains to the impaired sampling frame of the NSS. The NSC’s role in modernizing the NSS sampling frame has been ignored by both defenders and critics of the NSS in the recent debate on survey quality in India. An under-estimation of our urban population in NSS surveys has consistently figured in NSC meetings ever since it was set up, official records show.

In the early years of the NSS, the census was used as a sampling frame to pick samples in both rural and urban areas. After a few rounds, it became clear that the census database on urban settlements was inadequate. Given the fluid nature of many urban settlements and the lack of permanent house numbers, NSS staff had a hard time demarcating boundaries of sampling units in India’s towns and cities. A one-of-its-kind solution was found: a new survey to build an urban sampling frame for inter-censal years. The Urban Frame Survey (UFS) launched in the late 1950s was the kind of innovation for which the NSS was admired in its early years. It was a thoughtful and scientific solution to a desi problem. It greatly helped improve the accuracy of India’s official survey data.

As India’s statistical system weakened in the subsequent decades, so did the UFS. After the Rangarajan Commission flagged the issue of under-estimation of aggregate counts in NSS surveys, a panel was set up to examine this issue in 2005. Once the NSC was formed, it asked a former NSS chief, S.K. Sinha to look into this issue. In his report, Sinha argued that the undercount of urban population was largely due to a flawed UFS. The NSC then suggested that the UFS maps be digitized and integrated with the census maps. A lack of data sharing between different arms of the government hindered improvements in these databases, the NSC noted.

“The listing of households done by the census authorities and that, if any, done subsequently by local authorities is not available to NSS field functionaries even for comparison purposes,” the 2009-10 NSC annual report said. “...The population estimate emerging from the UFS of a town is never matched with the census population figure of the town. Number of UFS blocks in a selected town is a factor in the multiplier for urban areas, which contributes to underestimation in some measure.”

Over the next few years, the NSC appointed several committees to modernize the UFS database. Based on the recommendations of these committees, the NSC recommended NSS officials to integrate UFS and census area maps, and to use satellite data to update these maps in inter-censal years. Thanks to persistent prodding by the NSC, NSS functionaries published the UFS data and metadata in a user-friendly format in 2017. The latest UFS phase (2017-22) was supposed to be a fully digitized affair. How far it has been able to address past shortcomings will be known only when the results are compared with that of the next census.

The UFS modernization project was hobbled by the same issues that bedevil other statistical activities in the country: lack of resources, slow adoption of modern technologies, and lack of cooperation from other public agencies. An under-resourced NSC still managed to push the envelope. Imagine how much more effective it could have been if it were provided more teeth, and allowed to regulate all statistical work, including census operations.

Unfortunately, the recent debate on underestimation of the urban population in NSS surveys completely skirted issues of statistical governance. It followed a template that has become all too familiar now. A technocrat or a policy advisor discovers a legacy issue facing the statistical system, and offers loaded judgements. Statisticians and statistical advisors in turn offer knee-jerk and defensive responses, even when they know that the criticisms may not be entirely bereft of truth. Both sides avoid discussing substantive issues of statistical governance. The status quo persists till the next controversy blows up.

India’s once-vaunted statistical system was built by scholars and technocrats who debated statistical methods and governance issues constructively. They were quite careful in their writings and statements even while disagreeing with each other. We need a similar atmosphere of fair-minded discourse to rebuild India’s statistical edifice.