Home >Opinion >Columns >Opinion | Big data economists and their role in shaping the future

A recent article by CNN reporter Lydia DePillis says that Amazon has hired more than 150 PhD economists in the past few years. The online retailer is perhaps the biggest employer of economists in the US after the Federal Reserve. The key point is that these economists are at the centre of the action rather than detached advisers to senior management. “Amazon’s economists game out real estate decisions, set the lowest prices that will deliver a profit, precisely determine what customers care about and whether advertisements are working—all using machine learning algorithms that automate decision making on a massive scale," writes DePillis.

The growing role of economists in technology companies can be explained by the ubiquity of data as well as the power of computing. That is not all. Economists now have another role besides trawling through an ocean of customer data to find patterns. Economists have played a central role in the design of many online markets.

Just consider the way we use Uber compared to the way we use Airbnb. Uber has a centralized matching system. Its algorithm matches customers with cab drivers because we care more for a ride than the specific details of the taxi. The choice architecture is necessarily different when it comes to booking apartments for a vacation. The specific details of the house matters. So, Airbnb has a decentralized system of choice in which the customer rather than the algorithm picks the house.

There is something common as well. Both Uber and Airbnb have built ratings systems to build trust. Is the taxi driver known for rash driving? Is the stranger seeking to rent my place for a week notorious for leaving behind a trail of damages? Ratings help beat such information asymmetries—and create the trust that is essential in functioning markets. A lot of economic design has gone into these online markets.

Some of this has now begun to spill over into the realm of public policy. For example, The Billion Prices Project at the Massachusetts Institute of Technology (MIT) builds daily price indexes based on online prices. Its home page right now gives pride of place to a project to calculate the real inflation rate in Venezuela—where an economic collapse makes fudging of official statistics very likely. Economists working on the project spotted very early on that Argentina was under-reporting inflation in its official statistics, till a new price index was introduced in May 2016.

Indian public policy is also getting into the act. The Reserve Bank of India is setting up a data sciences lab. Economists at the finance ministry have already used big data analytics to chart our patterns of internal migration (using data from the railways computerised bookings system) and interstate trade (using preliminary data from the Goods and Services Tax Network). The Economic Survey released in August 2017 cited work by my colleagues at IDFC Institute based on satellite images to show how the density of built-up area in the Kozhikode Metropolitan Area spread between 1975 and 2014.

In a speech given at RBI in August 2018, Prof. Roberto Rigobon MIT Sloan School of Management drew a distinction between designed data and organic data. The former comes from surveys and administrative sources. The latter “is generated by individuals without them noticing they are being surveyed. It is the data in the GPS of your phone, your searches on the web, the friends in your network, the things you purchase". Organic data will begin to challenge the monopoly of designed data, though it cannot totally replace it.

Rigobon said that the main advantage of organic data is that it is truthful. It is based on actual behaviour rather than recall. Also, it is categorised based on behaviour rather than the traditional ordering by geography or socioeconomic conditions. The downsides are that organic data is often not representative and it can involve privacy violations.

Cut to October 2015. The Indian central bank was fighting a battle against high inflation. The committee of economists advising the RBI governor on monetary policy said in a statement released that month: “Moreover, there has been comfort on the inflation front—wholesale prices are contracting, GDP (gross domestic product) consumption deflator has been low at around 3%, and with vendors engaged in e-commerce offering low prices, retail inflation may be lower than what the headline number suggests" (emphasis added).

The growing use of digital transactions—by consumers, investors, tax payers—as well as the rise of newer forms of data collection has the potential to revolutionise Indian public policy. It is unlikely that these newer forms of data will completely replace the more traditional numbers derived from surveys, national accounts and administrative data. They will more likely complement each other. Government agencies will increase their dependence on big data analytics in the coming years—though the risks to individual privacy should not be underestimated.

Note: Here are three interesting papers on the topic.

1. Big Data And Measurement: From Inflation To Discrimination by Roberto Rigobon, Suresh Tendulkar Memorial Lecture

2. Peer-To-Peer Markets by Liran Einav, Chiara Farronato and Jonathan Levin, National Bureau of Economic Research

3. Economists (And Economics) In Tech Companies by Susan Athey and Michael Luca,Journal Of Economic Perspectives

Niranjan Rajadhyaksha is research director and senior fellow at IDFC Institute. Read Niranjan’s previous Mint columns at

Subscribe to Mint Newsletters
* Enter a valid email
* Thank you for subscribing to our newsletter.

Never miss a story! Stay connected and informed with Mint. Download our App Now!!

Edit Profile
My ReadsRedeem a Gift CardLogout