After decades of 'scientific guesswork', India Meteorological Department (IMD) is beginning to use supercomputer simulation-driven methods for monsoon forecasting in the last couple of years
New Delhi: In the middle of July 2017, roughly six weeks after the monsoon had set in, Gangabhishan Thaware, a farmer, walked into the Dindrud police station in Beed district of Marathwada with a peculiar complaint. He wanted to file an FIR against the Indian Meteorological Department (IMD) for “wrong monsoon forecast".
Monsoon rains had commenced by the end of May and after a brief spell of heavy showers in early June, the sky had turned completely dry for over a month, decimating Thaware’s cotton and jowar crop. “I lost nearly ₹ 50,000 per hectare," he said. “Marathi newspapers had published the monsoon forecast, predicting good showers for two months. Why are we spending lakhs of rupees on people and satellites to make these predictions," he asked.
The police ultimately did not register Thaware’s complaint. But his attempt to take the weather establishment head-on is a stark reminder of what happens when the realities of rural India come face to face with an inexact science and a less-than-ideal forecasting apparatus.
In contrast, the IMD this year clearly indicated a stalling of the monsoon by mid-June, signalling important changes that are underway in how India understands and predicts its monsoon—changes which are primarily driven by a massive expansion in supercomputing capacity.
Understanding the monsoon
The monsoon has kept its summer appointment with the subcontinent for at least the past 2 million years. Every year, between late-May and early-June, ocean winds travelling at an altitude of 20 km barrel into the African Highlands in Zimbabwe, get diverted towards India, and eventually get trapped between the Himalayas and the Burmese mountains. This sheer luck of geography turns India into some sort of a “monsoon bucket", said Raghu Murtugudde, an atmospheric science professor at the University of Maryland. But since the rains fall over such a big area, it is possible to distribute it in so many ways. “We don’t quite understand the dynamics of all of it," he said.
The result is that while the regularity of the monsoon is fairly certain, nobody knows how much of it will fall where, or when, over the course of the four months. Past experience is proving to be unreliable because the monsoon is undergoing fundamental shifts since the mid-1970s, resulting in a dramatic rise in the numbers of days of excess rain as well as no rain in certain regions, particularly central India.
The winds of change
In response, the Indian weather establishment, which has been heavily influenced by mechanisms put in place by the British, is finally shifting gears slowly. Until 2016, India’s monsoon forecasting system was driven entirely by statistical methods which tried to draw past correlation between monsoon rains and large-scale global phenomena, like the El Nino, and projected that into the present. “It was scientific guesswork," said S.K. Dash, a professor at IIT-Delhi. “You compare past rainfall data with, say, sea surface temperature. Then, make a scatter plot and whatever relationship is there, you presume it will happen in the future as well."
In the late-1800s and early 1900s, British meteorologists, exploiting the empire’s extensive geographic coverage, started noticing that several parts of the world experienced drought at the same time. The 1888 drought, in particular, which struck India and Australia at the same time, eventually established a relationship between warmer sea surface temperature in the Pacific Ocean (also known as El Nino) and rainfall failures across the southern hemisphere. India has used these large-scale global linkages, known as “teleconnections", to make predictions about the upcoming monsoon ever since.
The problem is that these global linkages can say almost nothing at scales below the size of a country. This didn’t matter to the colonial weather establishment, which had been set up around the time of a series of droughts in the late 1800s, collectively known as the late Victorian holocausts. Since India was economically important and absorbed a fifth of all British exports, a countrywide estimate on whether the monsoon would succeed or fail was adequate. It wasn’t necessary to know when there would be a heavy downpour over Mumbai or a dry spell in a village in Marathwada. This bias, which prioritized central planners over individuals, persisted for a long time.
“The models essentially treat the globe as a fluid... The equations describe atmospheric and oceanic flows. What it needs is today’s weather...- A.K. Mitra, scientist at National Centre for Medium Range Weather Forecasting
That has begun to change finally, after nearly a decade in the works, with India beginning to use supercomputer simulation-driven methods for monsoon forecasting in the last couple of years. These dynamical models simulate the state of the atmosphere and oceans at a particular time and then extrapolate into the future using standard laws of physics. “The models essentially treat the globe as a fluid," said A.K. Mitra, a scientist at the National Centre for Medium Range Weather Forecasting. “The equations describe atmospheric and oceanic flows. What it needs is today’s weather. You need to know today’s weather all over the globe. If you know ‘time zero’, then using physical laws, you can march ahead," he added. Unlike statistical methods, “these equations which are fed into the computer are scientific truth", Dash said. More importantly, these models can make predictions at smaller scales and, ideally, should be able to predict dry spells a month in advance. These month-long medium-term forecasts, which would kick in after the monsoon onset, are expected to begin experimentally from this year.
Globally, the shift towards simulations began 30-40 years ago and explains the disparity in weather forecasting capability between India and other countries. “India got its first supercomputer which had a memory of just 16 megabytes from the US in 1989," Mitra said. India entered into the teraflop (trillion calculations per second) club only recently.
The long-awaited ramp-up in computing capacity has now turned monsoon forecasting into one of the country’s most consequential emerging Big Data problems. In the first three weeks of May 2018, for example, India funnelled in 20 gigabytes (GB) of weather observations from across the world into several three-day long supercomputer simulation runs. Each of those runs incorporated 7 million datapoints that described the state of the atmosphere and oceans across the world in 3D at a particular moment.
But despite these seemingly big numbers, the skill of the dynamical model is still very low. As a result, the statistical method continues to remain the mainstay and only a small weight was assigned to the results thrown up by the computer simulations in both 2017 and 2018.
While India gets a large chunk of global weather data through mandatory systems put in place by the UN, there are large data gaps within the country. Almost no data is generated from the entire Himalayan mountain range
“We need more observations because if you cannot describe today’s weather accurately, then your future computations will go wrong," Mitra said. While India gets a large chunk of global weather data through mandatory systems put in place by the United Nations, there are large data gaps within the country. Almost no data is generated from the entire Himalayan mountain range. “Money was approved to set up some 140 Himalayan posts, but it could not be implemented," Mitra said. “There were logistical and tendering issues," he added.
India also has a strikingly low number of climate and weather scientists. Since the weather models in use were imported from the US and the UK and not home-grown, adapting and refining them to India’s tropical conditions would require a large pool of manpower and research.
Essentially, India’s weather prediction capabilities will remain poor without significant investments on three fronts—frequent supercomputing upgrades, an increase in weather observation data, and a significant boost in scientific manpower. Most foreign nations are working precisely on those three verticals since long-range weather prediction has now acquired strategic value, Mitra said. “There are at least 13 centres across the world which are also trying to predict the Indian monsoon. In a world where everything is traded, they would like to know before us," he added. The Central Intelligence Agency (CIA) of the US has also increasingly begun tracking weather from a food security and unrest perspective.
There are at least 13 centres across the world which are also trying to predict the Indian monsoon. In a world where everything is traded, they would like to know before us- A.K. Mitra, scientist at National Centre for Medium Range Weather Forecasting
Given these realities, it’s obvious that not enough resources are being put in to understand the monsoon, said Shreekant Gupta, an environmental economist at the Delhi School of Economics. Since we already know the monsoon is becoming more erratic, Gupta said that finance commissions should start factoring in environmental vulnerability into the devolution formula through a vulnerability index. “States vulnerable to climate risks should get more money," he added, which they can then use on mitigation measures with the help of more accurate district or block level rainfall forecasts.
But India is many, many years from that ideal scenario, said M. Rajeevan, secretary, ministry of earth sciences. “With current capability, beyond 3-4 days, it is not possible to predict exactly. The seasonal forecast is beyond predictability but we still make some prediction. As a result, out of 10 years, we may predict seven years correctly and may go wrong by a wide margin in the other three," he said.
This understanding of current limitations resulted in the push towards dynamical models and more computing resources in the last few years. “The whole world is working on improving these models," Rajeevan said. “Indian monsoon prediction is not at all a small problem with Indians now. It is a huge problem for the scientific and meteorological community in the world," he added.
The next step
Even as India embarks on the path of devoting greater resources, the monsoon itself is changing —from its onset date and intensity of rain to spatial spread—turning the whole attempt into a chase after a moving target. The global linkages (teleconnections) which invisibly bind the whole world, are also back in new avatars under the guise of climate change, inevitably pushing the country to understand newly emergent phenomenon far beyond its shores.
“The monsoon has always been like a turbulence," Murtugudde said. “We just have to keep on pushing our understanding," he said. But do the recent improvements in India’s forecasting capability allow the country to keep pace with the ongoing changes in climatic patterns? “Perhaps not," he said. “Seven billion people are changing the climate. It is hard for a few hundred scientists to keep pace with it." Clearly, more needs to be done.