Observer effect: Inside India’s media measurement meltdown

Barc has ordered a forensic audit. But this isn’t the first time such a scandal has hit the industry body responsible for measuring the viewership and reach of traditional TV. In 2020, then chief executive officer (CEO) Partho Dasgupta was arrested after allegations that Barc had manipulated ratings to favour the Republic network of news channels. The case was ultimately closed, but Barc was forced to suspend ratings for months.

The allegations assume gravity given that thousands of crores of advertising money are channelled to various media on the basis of their reach, measured by Barc.

Is TV the only medium where ratings, views and reach are tough to trust? Hardly. Media and marketing firm RK Swamy Hansa Group says brand recall on digital video platforms—including YouTube and Instagram—is “poor" at best. Most people can’t recall more than 1.5 brands from all the ads they see in an average of more than 2 hours of videos they watch per day, RK Swamy said in a white paper in September, after surveying 3,000 respondents across languages and platforms over a month.

Advertisers are pouring in money to run ads on Meta and Google’s networks, flooding banners on our TV news with their creatives, employing influencers on social media, and tracking an increasingly large number of metrics in the hope of discovering that Holy Grail for all marketers—RoAS or return on ad spends. Yet, the more complex media measurement gets, the more transparency seems to remain just out of reach. Why is it so? And what can be done to fix it?

Wolf at the door

Big Tech platforms have always been ‘walled gardens’, meaning their metrics on viewership, engagement, reach, and others cannot be verified by any third party. What Meta and YouTube tell us about their numbers have become the standard.

However, in the case of Barc, scandals involving the manipulation of measurement are doubly embarrassing, given that the ratings agency is owned by the broadcasters themselves.

“Foreign counterparts (of Barc) know that no BBC executive will go into suburban homes, offering £10 to watch ITV Channel 2 every day," a veteran media measurement industry said. “Barc, on its part, is setting up mechanisms constantly which are designed around policing rather than measurement. Obscene amounts of money are spent in making sure the measurement is not tampered with. This is tragic. Money should be spent on deepening and widening the research, not in trying to keep the wolf out of the door," the executive said, requesting anonymity citing their ongoing industry projects.

Following the Malayalam TV scandal, Barc said it “has immediately engaged a reputed independent agency to forthwith undertake a comprehensive forensic audit into the matter," and that it was committed to “the highest standards of integrity and accountability towards its stakeholders."

Barc did not respond to Mint's queries on why the system of sampling TV-watching households and measuring them remains vulnerable to manipulation.

One way to fix this problem proposed repeatedly over the years is to include more ‘meters’ in more homes to make sampling more robust. In 2022, the agency had said it will increase the number of meters to 65,000 and last year, Economic Times reported it had installed up to 59,000 meters already. But in a country with more than 190 million TV households, of which 30 million are connected or smart TVs, as per industry body Ficci and consulting firm EY, is even this upgraded sample size enough?

According to the government, no. This July, the ministry of information and broadcasting (MIB) published draft policy guidelines for television rating agencies in India, allowing multiple competing agencies to enter the ratings business, allow them to track both linear and connected TV, and proposing to increase the sample size of meters to 120,000, from the current 55,000 households.

“Installing new meters, maintaining them, measuring them and ensuring their confidentiality—all of this will cost a lot of money," a media measurement agency executive said. “Why will the broadcasters spend so much money doing so? Who will bid to become a TV ratings agency with this kind of upfront investment? These proposals have been floating about for a few years, but you can see how slow movement on it has been," the executive said on the condition of anonymity.

Incentives and fragments

This, perhaps, best underscores the problem: those in the business of publishing content do not seem to have an incentive to be honest about the metrics offered to users and advertisers. With Big Tech firms operating walled gardens, those incentives are skewed even further given how much they dominate the business of advertising and content altogether.

The other trouble, RK Swamy Hansa Group pointed out in its study, is how fragmented the media ecosystem has become altogether.

“Respondents struggled to recall specific brand names, often referring to ads by generic terms like “mobile ad"/“clothes ad," highlighting the difficulty in remembering and accurately identifying the brands. The most frequently recalled brands are associated with online shopping, online food ordering/delivery, groceries and coffee."

They identified two major issues why this may be happening repeatedly on platforms hosting online video. One, the digital video ecosystem has become too fragmented, ensuring no ad, platform, or piece of content gets a concentrated amount of attention as was the case with TV, radio, or other older forms of mass communication. Two, digital ads are very easily skipped, blunting recall even further. In fact, the RK Swamy survey found that 78% of all respondents skipped ads served during a video, while 50% muted ads if they couldn’t be skipped. Most telling, more than half the respondents said they found most ads on online video to be “irrelevant" and more than two-thirds found it irritating to see the same ad again and again.

“The performance metrics provided by various Digital Video platforms do not really throw much light on the impact of advertising on critical outcomes that determine ROI," the study finally concluded. “The fact [that] this media is becoming only more cluttered, fragmented and expensive to advertise on, is certainly not making things easier forcing many spenders to question what they are really getting."

In all this, advertisers and platforms are increasingly talking about building and using cross-platform measurement tools, including the MIB’s proposal to move TV ratings agencies to cover connected TVs too. Yet, there are huge gaps in this measurement ecosystem. Streaming platforms, for instance, are not covered by independent rating agencies the way TV is.

“While there have been attempts in the past to build integrated cross-media measurement in India, including those by Barc India, which handles television ratings in India, there hasn't been any meaningful progress, as different platforms have different approaches to what metrics are important, and what data should be shared publicly," Shailesh Kapoor, founder and CEO of research and measurement firm Ormax Media told Mint.

“Advertisers who are advertising on OTT demand data and analytics, which platforms are providing currently too. An integrated, central source is ideal, but the technological and ideological challenges involved are huge barriers. So we may not see a solution anytime soon."