Google refines AI search overviews after odd results

The Alphabet unit was prompted to pull back on the new feature after users reported strange and incorrect answers, such as promoting rock consumption for health benefits and using glue to keep cheese sticking to pizza.

AI overviews, launched earlier this month, provide users with a passage of text answering questions at the top of Google search results before the list of links. Google said it was able to observe patterns where the problems occurred and made a dozen technical improvements to the feature.

“We hold ourselves to a high standard, as do our users, so we expect and appreciate the feedback, and take it seriously," Liz Reid, a vice president overseeing Google search, said in a blog post published Thursday.

The response is the second time this year that Google has had to publicly course correct after a major AI release. In February, Chief Executive Officer Sundar Pichai said the company would make structural changes after Gemini, a chatbot based on Google’s most advanced AI technology, angered users by producing ahistoric images and blocking requests for depictions of white people. The controversy morphed into a broader backlash against the chatbot’s responses to different philosophical questions.

Google is embracing generative AI, a type of software that can produce text and other forms of media, in order to fend off growing competition to its core business. The rollout of AI overviews is viewed as an important step in the company’s defense of its search market, where it makes hundreds of billions of dollars in advertisements.

The search giant said it extensively tested AI overviews before they were launched. “But there’s nothing quite like having millions of people using the feature with many novel searches," Reid said in the blog post. “We’ve also seen nonsensical new searches, seemingly aimed at producing erroneous results."

Google also said there was a large number of fake screenshots pretending to show results from its AI overviews and encouraged anyone encountering screenshots showing dangerous results to do a search themselves to check.

In response to the odd, inaccurate or unhelpful overviews that showed up, Google has built a better detection system for determining nonsensical queries that now won’t show the AI-generated results. It also limited user-generated content that could offer bad information.

Reid said user feedback shows that with AI overviews, people have higher satisfaction with their search results. She said that Google would continue making improvements to the feature and strengthen protections. “We’re very grateful for the ongoing feedback," she wrote.

Write to Aaron Tilley at aaron.tilley@wsj.com