BackPushing evidence-based policymaking for the poor
Abdul Latif Jameel Poverty Lab and its collaborators have shown the transformative potential of harnessing scientific rigour for development objectives in India
Premium
We set up J-PAL, the Abdul Latif Jameel Poverty Action Lab, at the Massachusetts Institute of Technology (MIT) in 2003. J-PAL is now a network of researchers around the world (160 and counting) united by the mission to improve the quality of the policies that affect poor people worldwide by ensuring that policy decisions are based on scientific evidence. Our tool of choice is the randomized controlled trial (RCT), borrowed from medical science: each candidate’s intervention is first tried out in a treatment group randomly chosen from a larger population. In this way, any difference between the treatment group and the comparison group can be confidently attributed to the treatment. So for example, to study the effect of a new teaching tool, 100 schools chosen by lots from 200 will get the tool, while the rest will continue to use the status quo methods to see whether the new pedagogy makes any difference.
Today, J-PAL researchers have conducted or are in the process of conducting about 850 distinct RCTs in close to 80 countries. The research spans health, nutrition, healthcare, education, finance, labour markets governance, the environment and the design of social support programmes. This year we are celebrating the 10th birthday of our South Asia office and today, India has by far our largest office.
The setting up of an office for J-PAL South Asia at the Institute for Financial Management and Research (IFMR) in India was, for the most part, just putting a stamp on what was a fait accompli: by 2007, J-PAL’s biggest single operation was already in India with 21 ongoing or completed projects. India is fortunate to have some of the brightest and most thoughtful policymakers in the world, both inside and outside the government. A lot of our ability to carry out large-scale experimental studies of policy interventions relied crucially on their willingness to hear us out and give us a chance to demonstrate the many benefits we were claiming for the experimental approach to policy.
We did not fully realize it then, but we were in fact very lucky with our early partners. Our first partners in carrying out field experiments were Seva Mandir and Pratham, two organizations that are unlike each other in every way except in being exceptional. Seva Mandir was an old Gandhian organization that worked and still works in and around Udaipur district, trying to build effective community-based development. In 1996, Ajay Mehta and Nilima Khetan, who then led Seva Mandir, agreed to carry out what we believe was India’s first RCT in the social sciences. This was at a time when most people who heard the idea were convinced we were mad scientists in lab coats who thought of people as giant mice.
Seva Mandir, then as now, was an organization that suffered a bit from a version of the Groucho Marx problem—no standard that they could realistically meet was high enough for them, there was always room for self-criticism and aiming higher. They wanted to do the right thing, and to do it in the right way, for the right reason. They believed that the experiment they were getting into with us would simply confirm that they were doing the right thing. But when our experiment showed that the addition of an extra teacher to a one-teacher school had no effect whatsoever on what got taught or what the children learnt in their schools, in part because the presence of a backup teacher encouraged delinquency, instead of telling us that we were getting something wrong—which would not have been unfair given that they were old hands at programme implementation while we were greenhorns at this kind of research—they went headlong into self-examination.
The result was more research, another RCT, this time on how to get teachers to come to school and whether that matters for learning. This was in the days when incentives were a dirty word in education, and for Seva Mandir, as an old Gandhian organization committed to moral rather than financial rewards, not at all an obvious thing to do. But when the results showed that incentives worked and helped children learn more, they were the first to embrace the results, which were not always popular with their friends.
If Seva Mandir’s strength and occasionally, weakness, was their profound distaste for half-measures, Pratham thrived on ignoring the many education experts who accused them of only scratching the surface of the problem. They were keenly aware that the problem of providing quality education for all of India’s children was immense—so big indeed that no one yet had even started to quantify it in the way that Pratham eventually would through its remarkable Aser (Annual State of Education Research) surveys. It was easier to pontificate about what the ideal democratic education system should look like than to think hard about exactly how one would get anywhere close to that system at the scale needed to be relevant. There were, after all, many tens of millions of children (and indeed after Aser we know the number is closer to a hundred million) who had spent several years in school and could not yet read.
From the very early days, Pratham took the view that the only way we would get there is by trying out different ideas, in part to find some levers to move this mountain, and in part as a diagnostic tool. This experimental approach to running their organization made us their natural partners, though it took us some time to persuade Madhav Chavan and especially Rukmini Banerji, the moving spirits, then and now, behind Pratham, that we were worth taking seriously. Since then of course, Pratham and our J-PAL colleagues (including us) have worked together on a dozen or so RCTs. Out of that (and the many conversations we have had on the sidelines of an RCT) has emerged a simple but powerful theory of why schools fail.
Ultimately, we believe, everyone in the education system is focused on the few children who, from the early days, are marked for success. The tyranny of the syllabus does not allow most teachers to step back and help the children who have fallen behind catch up, though in the rare cases when they do, there are dramatic positive results. This is the idea of teaching at the right level (TaRL), which Pratham and J-PAL now jointly promote to governments all over the world.
Not all our great partners were in the voluntary sector. Nachiket Mor, who now runs Gates Foundation in India, but in the early 2000s was part of the leadership team at ICICI Bank, encouraged us from the very early days to do rigorous work on the impact of financial products with his combination—as anyone who knows Mor will instantly recognize—of brilliant insights, unexpected questions and insatiable curiosity. In particular, he helped us very directly to overcome the resistance that we faced from the microcredit industry to any evaluation of their products. Even though much of our work did not find what he might have hoped for—our results suggested that neither microcredit and micro-insurance, two products that ICICI bank was then pushing, play the kind of transformative role that people expected from them—Mor has always stood by us and our commitment to what the data shows.
Since then, thanks in part to our connections with these wonderful people and the success of our work with them, we have had an easier ride. It is no longer considered insane to want to do an RCT (though many people still have good and bad objections to them). Our scale has grown: J-PAL South Asia has 70 projects ongoing right now, and another 102 are already completed. There are nine offices across India—our head office is at IFMR in Chennai, but the biggest office happens to be in Delhi. There are 175 people working across these offices on a regular basis and, at any given time, over a thousand field officers are busy collecting data. We are currently working with 16 state governments and several line ministries in the Central government.
Perhaps more unusually, several top bureaucrats and police officials—such as Nina Singh of the Rajasthan police (and now CBI), Girija Vaidyanathan and S. Krishnan (chief secretary and planning secretary, respectively, of Tamil Nadu), Hardik Shah (Gujarat Pollution Control Board) and Santhosh Mathew (then at the ministry of rural development)—have been deeply involved in research design. Many of them consented to be our co-authors on research projects and papers, bringing in entirely novel insights while keeping us focused on what really matters.
These relationships reflect a small revolution in how economists do their research. The research is hands-on rather than hands-off, solving real problems but also learning better how the world works. The traditional dichotomy between the starry eyed researcher trapped in the ivory tower and the practically minded implementer who is too busy to reflect is crumbling. Good researchers know how to be practical and good policymakers know when they need to move beyond their comfort zone, and they relish the experience.
Beyond each individual project or collaboration, what gives us real hope for the future of policymaking in India is to see, in the corridors of power, as in the makeshift offices of NGOs and in press rooms all over the world, not only a willingness but often even a desire to engage with facts, even uncomfortable facts. We need this to counter the temptation, ever present at the top, to govern through slogans and ideologies, and to make policies for a country and people that only exist in the mind.
Abhijit Banerjee and Esther Duflo are, respectively, co-founder and director of the Abdul Latif Jameel Poverty Action Lab and Ford Foundation international professor of economics at MIT, and co-founder and director of the Abdul Latif Jameel Poverty Action Lab and professor of poverty alleviation and development economics at MIT.
This is the first in a series of three articles tracing the efforts of J-PAL to promote evidence-informed decision making in India. In the second article, Iqbal Dhaliwal will discuss the potential for evidence-informed decision making in Central and state governments and J-PAL’s experience working with some such policymakers; and in the third article Shobhini Mukerji and Rukmini Banerjee will dive deeper into a specific case study, using rigorous evidence to understand the drivers of learning outcomes in schools and scale up programmes based on that research.
Comments are welcome at views@livemint.com
Unlock a world of Benefits! From insightful newsletters to real-time stock tracking, breaking news and a personalized newsfeed – it's all here, just a click away! Login Now!
