Using evidence for better policy: case of primary education in India10 min read . Updated: 20 Mar 2013, 04:25 PM IST
While India has achieved some success in increasing primary school enrolment learning outcomes continue to be abysmally low
While India has achieved some success in increasing primary school enrolment learning outcomes continue to be abysmally low
Investments in education contribute to aggregate economic growth and enable citizens to participate in the growth process through improved productivity, employment, and wages, and are, therefore, a critical component of the inclusive growth agenda of the Government of India. The past decade has seen substantial increases in education investments under the Sarva Shiksha Abhiyan (SSA), and this additional spending has led to considerable progress in improving primary school access, infrastructure, pupil-teacher ratios, teacher salaries, and student enrolment.
Nevertheless, student learning levels and trajectories are disturbingly low, with nationally representative studies showing that over 60% of children aged 6-14 are unable to read at second-grade level. Further, learning outcomes have shown no sign of improving over time (and may even be deteriorating). Thus, the poor performance of the education system in translating spending into outcomes threatens aggregate productivity in the economy and also denies citizens the capabilities they need to fully participate in a modernizing economy.
The past decade has also seen a number of high-quality empirical studies on the causes and correlates of better learning outcomes based on large samples of data and careful attention paid to identification of causal relationships. This research has yielded robust findings both on interventions and inputs that do not appear to contribute meaningfully to improved education outcomes, as well as on interventions that are highly effective.
In particular, the research over the past decade suggests that increasing inputs to primary education in a business as usual way is unlikely to improve student learning in a meaningful way unless accompanied by significant changes in pedagogy and improvements in school governance. It is, therefore, imperative that education policy shift its emphasis from simply providing more school inputs in a business as usual way and focus on improving education outcomes.
The most important components of education spending in the past decade have been on improving school facilities and infrastructure, improving teacher salaries and training, hiring more teachers to reduce pupil-teacher ratios, and expenditure on student benefits such as textbooks and mid-day meals. Analysis of both administrative and survey data shows considerable improvements in most input-based measures of schooling quality. But the research of the past decade finds very little impact of these improvements in school facilities on learning outcomes (this is true across multiple studies and data-sets as summarized in Muralidharan, K.(2013): “Priorities for Primary Education Policy in India’s 12th Five-Year Plan," India Policy Forum.) This is not to suggest that school facilities and infrastructure do not matter for improving learning outcomes (they may be necessary but not sufficient), but the results highlight that infrastructure by itself is unlikely to have a significant impact on improving learning levels and trajectories. Similarly, while there may be good social reasons for mid-day meal programmes (including nutrition and child welfare), there is no evidence to suggest that they improve learning outcomes.
Even more striking is the fact that no credible study on education in India has found any significant positive relationship between teachers possessing formal teacher training credentials and their effectiveness at improving student learning. Similarly, there is no correlation between teacher salaries and their effectiveness at improving student learning, and at best very modest positive effects of reducing pupil-teacher ratios on learning. As discussed further below, these very stark findings most likely reflect weaknesses in pedagogy and governance which are key barriers in translating increased spending into better outcomes.
The results summarized so far can be quite discouraging, and could plausibly be interpreted as suggesting that “improving learning outcomes—especially across a distribution that includes millions of first-generation learners—is very difficult, and so the best we can do is to provide the standard inputs associated with functioning schools and hope for positive effects in the long run". Fortunately, the news is not all bad, because the evidence of the past decade also points consistently to interventions that have been highly effective at improving learning outcomes, and are able to do so in much more cost-effective ways than the status-quo patterns of spending.
While there have been significant increases in schooling inputs, a key determinant of how these investments translate into learning outcomes is the structure of pedagogy and classroom instruction. Getting aspects of instruction right is particularly challenging in a context such as India where several million first-generation learners have joined a rapidly expanding national schooling system. In particular, standard curricula, text books, and teaching practices that may have been designed for a time when education was more limited may not fare as well under the new circumstances, since the default pedagogy is one of “completing the textbook", which increasingly does not reflect the learning levels of children in the classroom, who are considerably further behind where the textbook expects them to be.
Evidence that ‘business as usual’ pedagogy can be improved on is found in several randomized evaluations finding large positive impacts of supplemental remedial instruction in early grades that are targeted to the child’s current level of learning (as opposed to simply following the textbook). Four points are especially noteworthy. First, these positive results have been found consistently in programmes run by multiple non-profit organizations in several locations (including Uttar Pradesh, Bihar, Uttarakhand, Gujarat, Maharashtra and Andhra Pradesh).
Second, the estimated magnitudes of impact from these interventions (whose instructional time is typically only a small fraction of the duration of the scheduled school year) are considerable - often exceeding the learning gains from a full year of conventional schooling. Third, these interventions are typically delivered by modestly paid community teachers, who mostly do not have formal teacher training credentials. Finally, these supplemental remedial instruction programmes are highly cost-effective and deliver significant learning gains at much lower costs than the large investments in the standard inputs (reviewed above) that have not been found to be effective.
Beyond pedagogy, another explanation for the low correlation between increases in spending on educational inputs and improved learning outcomes may be the weak governance of the education system and limited effort on the part of teachers and administrators to improve student learning levels. The most striking symptom of weak governance is the high rate of teacher absence in government-run schools. While teacher absence rates were over 25% across India in 2003, a survey that covered the same areas in 2010 found that teacher absence in rural India was still around 24%. The cost of teacher absence was estimated at around ₹ 7,500 crore per year, suggesting that governance challenges remain paramount in the education system.
On the positive side there is evidence that even modest improvements in governance can yield significant returns. The all-India panel data show that improving monitoring and supervision of schools is strongly correlated with reductions in teacher absence, and we estimate that investing in improved governance by increasing the frequency of monitoring could yield an 8 to 10 times return on investment in terms of reducing the cost of teacher absence and could be 10 to 12 times more cost-effective at reducing effective pupil-teacher ratio (which is the pupil-teacher ratio after adjusting for teacher absence rates) than hiring more teachers.
The evidence also points to the importance of motivating teachers by rewarding good performance as a key lever in improving the performance of the education system. Rigorous evaluations of carefully designed systems of teacher performance pay show substantial improvements in student learning in response to even very modest amounts of performance-linked pay for teachers (that was typically not more than 3% of annual pay). Long-term evidence over five years in Andhra Pradesh shows that teacher performance pay was 15 to 20 more times more effective at raising student learning than reductions in pupil-teacher ratios, which is a default policy position for improving education quality. More broadly, these results suggest that the performance of frontline government employees depends less on the level of pay and more on its structure. In particular, introducing small amounts of performance-linked pay is much more likely to improve public worker performance than large amounts of across- the-board increases in pay, and is also much more cost-effective. The lessons from this research are likely to be relevant not just for teachers but for government employees more broadly.
From evidence to policy
The evidence summarized above does not imply that we should stop improving school infrastructure or training teachers. Rather, it strongly cautions that more of the ‘business as usual’ expansion of education spending is unlikely to solve the crisis in learning outcomes. It also highlights the critical importance of considering evidence on cost effectiveness in making optimal policy in a fiscally constrained environment.
Three immediate policy implications of this research are summarized below:
1) Make learning outcomes an explicit goal of primary education policy and invest in regular and independent high-quality measurement of learning outcomes: A truism of management in large organizations is that “what you measure is what you get". The Indian state has done a commendable job in improving the education indicators that were measured (including school access, infrastructure, enrolment, and inclusiveness in enrolment) but has fallen considerably short on the outcome indicators that have not been measured (such as learning outcomes). While independently measuring and administratively focusing on learning outcomes will not by itself lead to improvement, it will serve to focus the energies of the education system on the outcome that actually matters to millions of first-generation learners, which is functional literacy and numeracy (that the system is currently not delivering).
2) Launch a national campaign of supplemental instruction targeted to the current level of learning of children (as opposed to the textbook) delivered by locally hired teacher-assistants, with a goal of reaching minimum absolute standards of learning for all children: While gaps in enrolment between disadvantaged groups and the population averages have decerased, there is a considerably larger gap in learning levels, which exists at the point of entry into the school system and continues to widen over time. Thus, the gains of the past decade made in terms of reducing inequities in primary school enrolment will be at considerable risk (because low learning levels are strongly correlated with the probability of dropping out) if urgent attention is not paid to the crisis in learning outcomes with a mission-like focus on delivering universal functional literacy and numeracy that allow children to “read to learn". The evidence strongly supports scaling up supplemental instruction programmes using locally hired short-term teaching assistants that are targeted to the level of learning of the child, and the cost-effectiveness of this intervention also makes it easily scalable.
3) Pay urgent attention to issues of teacher governance including better monitoring and supervision as well as teacher performance measurement and management: A basic principle of effective management of organizations is to have clear goals and to reward employees for contributing towards meeting those goals. The extent to which the status quo does not do this effectively is highlighted in the large positive impacts found from very modest improvements in the alignment of employee rewards with organizational goals. Implementing these ideas effectively in a public sector setting will take considerable effort, but the evidence highlights the potentially large returns in doing so.
The next 10 years will see the largest ever number of citizens in the school system at any point in Indian history (or future), and it is critical that this generation that represents the demographic dividend be equipped with the literacy, numeracy and skills needed to participate fully in a rapidly modernizing world. In a fiscally constrained environment, it is also imperative to use evidence to implement cost-effective policies that maximize the social returns on any given level of public investment. The growing body of high-quality research on primary education in the past decade provides an opportunity to put this principle into practice.
Karthik Muralidharan is with the University of California, San Diego. A shorter version of this piece appeared as a section of Chapter 2 of the Economic Survey of India 2013. This column is based on the author’s background paper on primary education policy for the 12th Five-Year Plan that summarized a decade of research and synthesized the policy implications.
This column has been reprinted with permission from Ideas for India, www.ideasforindia.in-