Should you trust that five-star rating on Airbnb?

The short answer is yes: better to have some information than none. But the flaws of online reviews are evident. For products with some objective measures of quality, there is a big gap between the views of punters and experts. A study in 2016 by Bart de Langhe of Vlerick Business School, in Belgium, and his co-authors found that user ratings for 1,272 items listed on Amazon.com bore little relation to either the verdict of Consumer Reports, an American product-testing organisation, or to their resale value.

That might be because consumers place greater value on more subjective things like a product’s brand. But if ratings are based on subjective criteria, then another problem arises: what if your tastes differ from other people’s? The best book ever, according to members of GoodReads, an online community of bibliophiles, is “The Hunger Games" by Suzanne Collins. You may agree, but plenty of people do not.

Another problem is that the people who bother to leave reviews and ratings may not be representative of consumers as a whole. In a study published in 2020, Verena Schoenmueller of Esade, a business school in Spain, and her co-authors examined the distribution of ratings left in around 280m reviews of more than 2m products and services on 25 different platforms. They broadly confirm a familiar pattern: a polar distribution of ratings, with more of them at the extremes of the scale than in the middle, and a skew towards more positive ratings.

There are lots of theories as to why online reviews follow this pattern. People who have chosen to buy something are already more likely to be satisfied with it. Extreme experiences, good and bad, are more likely to prompt reviews. Some write-ups are not real: estimates of the prevalence of fake reviews vary but they are certainly a problem, and one which generative AI may make worse.

The type of platform matters, too. Sharing-economy markets have a different feel. You could leave a four-star review for your Airbnb stay, but now that you have established a relationship with the hosts, and since they are also rating you, it’s much easier to just award five. A paper by Georgios Zervas of Boston University and his co-authors, last updated in 2020, found that average ratings for Airbnb properties are consistently higher than those for hotels on Tripadvisor.

In theory, businesses have an interest in soliciting as representative a sample of reviews as possible. Honest customer feedback is the best way to spot and fix problems, after all. In practice, the importance of good ratings, particularly for firms that are struggling for visibility, is an incentive for jiggery-pokery. A study from 2013 by Dina Mayzlin of the University of Southern California and her co-authors suggested, for example, that small, independently owned hotels generated more positive fake reviews on Tripadvisor than branded hotel chains.

If the incentives of businesses and consumers do not always align, then platforms have an interest in ensuring that reviews are as informative as possible. Weighting scores by the number of reviews that a customer writes could help mitigate the problem of polarity; Ms Schoenmueller’s research suggests that the more reviews a person writes, the less extreme their ratings.

But consumers can also help themselves. Mr de Langhe’s research suggests that people put too much weight on the overall average rating. The absolute number of reviews is a better indicator of actual popularity. And it is the detail in a review that tells you whether the person writing it prefers dystopian young-adult fiction to other genres, or whether a diner values buzz or the ability to hear themselves think. Reviews, then, are even more useful if you read them.

Subscribers to The Economist can sign up to our Opinion newsletter, which brings together the best of our leaders, columns, guest essays and reader correspondence.