Opinion | Randomized control trials may not suit India’s social schemes4 min read . Updated: 31 Oct 2019, 10:46 PM IST
What works for a small-scale NGO-style intervention may not help the state’s implementation of it without elaborate checks
The Nobel Prize for economics this year has gone to three scholars, two American citizens and one French-American. It has generated much excitement in India because one of the Americans, Abhijit Banerjee, is of Indian descent, and all three have worked on India. This has happened before. Angus Deaton, the 2015 recipient, and Amartya Sen (1998), had done research on India too.
The difference this year is that the randomized control trials (RCTs) used by the awardees gave their work a sharply prescriptive aspect, following the example of the pharmaceutics field, where a vaccine or medication once validated through an RCT is approved for prescription. Inducted into economics, whether RCTs yield actionable results is a function of study design: the framing of the hypothesis, the composition of the control group, and the context in which the study is done. The last two are especially important because the placebo control in medical research is not so easy to implement in behavioural research. I have a published research paper, with co-author Manish Gupta, in which we establish that the findings in a paper by Duflo and an Indian co-author, Raghbendra Chattopadhyay, are based on an incorrect framing of the hypothesis being tested.
The citation for the Nobel award reads: “In just two decades, their new experiment-based approach has transformed development economics, which is now a flourishing field of research." RCTs in economics carry very high research costs. This has led to the corporatization of economics research.
RCT papers are lodged in a silo with a high incidence of mutual citation. As journals in any field look for a high impact factor, which is a function of citation incidence, the best development economics journals now have a marked preference for RCT papers. This has skewed published research and pushed out other approaches, which might have yielded far more useful results.
What the RCT body of work does establish beyond a shadow of doubt is that people in India respond in a predictable way to incentives. Parents are shown in one study to complete a child immunization course if motivated by a reward at the end, such as metal meal plates.
What works for a small-scale NGO-style intervention may not help in implementation by the state without an elaborate system of checks on whether people are indeed getting their children vaccinated or just making false claims. Effective monitoring is not a known strength of the administrative system in India.
However, there is a further problem with personalized rewards like metal utensils. Immunization, or school-going for that matter, has to be part of a universal drive, where joining has to be incentivized as a badge of belonging to the community. That was what worked for the successful polio campaign in India, in partnership between the World Health Organization and the Indian government. A flat-out television campaign with popular film stars such as Amitabh Bachchan made for a community spirit that ensured that no child was left out.
Another RCT study in Africa treated the water supply of randomly selected villages to get rid of parasitical worms, and tested for whether the children in this treated group of villages did better on school scores. There is a serious ethical issue here. If the study is underpinned by a known technology for eliminating worms from drinking water, it is imperative that all villages be treated without exception, regardless of whether school scores improve as a result.
After the Nobel award, Delhi chief minister Arvind Kejriwal publicly thanked RCTs for having underpinned the chunauti (challenge) scheme, which led to better performance of students in Delhi state schools. What matters is the level at which the incentive is set—at the team level (such as for the school as a whole), or at an individual teacher level. If at the latter (as recommended in some RCT studies I have seen), there will be intra-school squabbling, rendering the school itself totally dysfunctional.
What economics taught much before RCTs is that incentives, whether explicit or implicit, critically shape outcomes. If there is a loan mela with targets set, bank officials at the last mile will sacrifice due diligence to meet their target. Or, in routine lending, if bank officials fear that a decision made in good faith could later be questioned for complicity in fraud, they will be incentivized to prefer inaction to action. In countries with effective governance, these problems are resolved by assigning decision-making to teams of manageable size, where one team can present its decisions to other teams for advice or assessment. There is no room for RCTs in any of this.
If the Nobel committee was looking for someone who “has considerably improved our ability to fight global poverty", it should instead have chosen Jean Dreze, an Indian citizen of Belgian descent. He was instrumental in designing the Mahatma Gandhi National Rural Employment Guarantee Scheme (MGNREGS), which bypasses the problem of poverty-targeting through self-selection. He was also instrumental in giving it an administrative architecture, which has successfully delivered results on the ground.
Indira Rajaraman is an economist.