BackOpinion | Rankings cannot reveal the health of systems
A systems approach is necessary to improve the Ease of Doing Business, particularly to aid the small enterprises
Premium
India has climbed an impressive 55 rungs in the World Bank’s Ease of Doing Business rankings in two years. The Narendra Modi government must be congratulated because such global rankings improve India’s image abroad. The rankings do not reflect conditions on the ground, many commentators say. They are right. Because rankings can hide internal conditions. Rankings require a single number, whereas the health of complex systems cannot be measured with any one number. Many variables must be considered. All of them must perform adequately.
Consider the human body. Good health requires that all subsystems perform well—cardiovascular, digestive, nervous, muscular, etc. If any of the vital systems is weak, the overall health is poor, even if all the other systems are functioning well. A single number computed from ratings of inter-related subsystems can produce strange results. For example, if four of five of the critical systems get 10 on 10, and one of them is a zero, the overall score will be 8. Which seems very good, except that the patient is actually in a critical condition. In comparison, if all five systems get 7 each, the overall score will be 7. This patient, though much healthier than the first, will rank lower overall. Therefore, a drive to improve rankings by focusing on only some subsystems while the rest remain in poor shape will not improve the actual health of the system.
This is the conceptual flaw in the World Bank rankings’ approach for improving the ease of doing business. Accusations of manipulation of the rankings are irrelevant. Those are distractions, about the placements of the deck chairs on the Titanic, whereas the problem is more fundamental. The independent panel set up by the World Bank president in 2012 to evaluate its methodology recommended changes to make the analysis more useful for countries, downplaying rankings. Its recommendations were set aside. Rankings attract attention to the World Bank’s flagship Ease of Doing Business report. The annual rankings give the World Bank much-desired publicity, whether or not they are very useful to the countries for making real improvements.
Producing a single number from measurements of the subsystems requires relative weights to be assigned. In the example above of the human body, all subsystems are given equal weight. Some may argue that if different weights were given, the result would be a better reflection of the system’s health. Which begs the question, of course, of what should be their relative weights in the performance of the whole?
The hazards in assigning weights are illustrated by the difficulty the US Air Force has had in selecting a replacement for its ageing fleet of refuelling tankers. The selection process was stalled for many years by ethics scandals and legal challenges by losers whenever the US Air Force announced its decision. (The Indian Air Force seems to be suffering similarly too!) Therefore, US authorities decided to remove all subjective elements in the evaluation. A total of 373 mandatory requirements were laid out, each of which had to be met in a simple pass-fail test. In this effort, to avoid any bias in the weighting, the water flow in toilets is rated as highly as the fuel offload rate!
Pradeep Mehta has given a good analysis (‘Improving the Business Climate’, Mint) of the disjunction between the improvement of India’s rankings in the World Bank assessment and the realities on the ground. He points out that India’s rankings have improved with great improvements in procedures for obtaining construction permits, whereas improvement in other areas that may matter as much or more to small enterprises, such as the ease of obtaining credit and enforcing contracts, remain big problems.
The World Bank’s methodology makes these aspects ‘easy to rank’. It does not translate well into ease of doing business. Pictures of the shape of the system, such as a ‘spider diagram’ in which the conditions of all critical variables are shown in the same picture, would reveal more. Very fast progress on one or two subsystems, with limited progress on the others, will show a lopsided picture in a spider diagram—an indication of ill-health of the system. Consider again the example of human health mentioned before. The spider diagram of the health of the body with 7s in all variables will be more rounded. But the other diagram, with high numbers on some and very poor numbers on others will point to the areas that need attention, rather than suppressing them within an overall indication of high achievement.
Rankings are useful for ‘naming and shaming’, though not useful for focusing attention on areas needing improvement. While the independent panel was preparing its report on the World Bank’s Ease of Doing Business project, the erstwhile Indian Planning Commission examined conditions within the country. The Planning Commission recommended a systems approach, rather than a ranking approach, to assist states to improve conditions on the ground, particularly for small enterprises that generate the most employment. It recommended the use of techniques such as spider diagrams to make the overall health of states’ systems visible, and to point to areas needing improvement. Such a systems analysis, if it is being done now, seems to have been overtaken by the very visible ranking of states.
The health of a complex system cannot be gauged from a single number. The shape of a system matters, not just its size. The size of the GDP, the number that attracts great attention as a surrogate of a country’s development, does not reveal the deterioration of the environment, or the inequalities in society, caused by the blinkered pursuit of GDP. Replacing GDP with a singular ‘happiness index’ or ‘sustainability index’ will no doubt broaden what is measured. But why is a single number required? How will it help a country know what is going on inside it? It will be more useful to present some form of ‘balanced’ score card, such as a spider diagram, as an x-ray to reveal the conditions of all critical subsystems.
Arun Maira was a member of the erstwhile Planning Commission.
Comments are welcome at theirview@livemint.com
Unlock a world of Benefits! From insightful newsletters to real-time stock tracking, breaking news and a personalized newsfeed – it's all here, just a click away! Login Now!
