Google Eats Glue-Laced Humble Pie Over AI-Powered Search Results
Left image 77759465 | Logo © Bennymarty | Dreamstime.com, right image generated on AI
To glue or not to glue? That was the question users mulled over after Google’s latest feature, AI Overviews, recommended adding “non-toxic glue” to prevent cheese from sliding off pizza. Launched at Google I/O 2024, this addition promised to provide concise, artificial intelligence-driven summaries and answers to user queries. The reality was that there were several hits and misses, which have brought the feature’s reliability into question. The tech giant has been forced to scale back the functionality as a result.
As detailed by Liz Reid, vice president and head of Google Search, in a lengthy response, AI Overviews is distinct from the usual chatbots and language models as it integrates directly with Google’s core web ranking systems. It’s engineered to perform typical search tasks like identifying relevant, high-quality results from Google’s vast index.
r/Whoosh
Where it fumbles is at identifying sarcasm, which, as you know, is rife across the wild, wild web. In the case of Pizzagate, AI Overviews pulled from a Reddit forum and guided users to “add about 1/8 cup of non-toxic glue to the sauce to give it more tackiness.”
Google AI overview suggests adding glue to get cheese to stick to pizza, and it turns out the source is an 11 year old Reddit comment from user F*cksmith 😂 pic.twitter.com/uDPAbsAKeO
— Peter Yang (@petergyang) May 23, 2024
One other high-profile blunder involved the AI incorrectly stating that Barack Obama was the first Muslim president of the United States, referencing a debunked conspiracy theory. It also told people to infuse spaghetti with gasoline for flavor and to use more oil to extinguish a fire.
Responding to the query, “How many rocks should I eat?” Google cited a satirical article on a geological software provider’s website. In another instance, it wrongly detailed the number of moons orbiting Mars.
2. Smoking While pregnancy pic.twitter.com/4s6yy9McVG
— Hamza Khalid (@Whizz_ai) May 30, 2024
These gaffes usually arise from the AI misinterpreting queries, nuances in language, or a lack of quality information.
Google has now addressed the criticisms by taking several steps to improve the feature. The company has limited the inclusion of satire and humor content, updated its systems to reduce the use of user-generated content in responses, and added restrictions for queries where AI Overviews have proven unhelpful.
The company asserts that despite the tool being designed to “hallucinate” less, “some odd, inaccurate or unhelpful AI Overviews certainly did show up,” Reid acknowledges. This primarily happens with “queries that people don’t commonly [make],” such as unserious questions.
For example, “practically no one” had asked how many rocks they should eat on the traditional search engine. “This is what is often called a ‘data void’ or ‘information gap,’ where there’s a limited amount of high quality content about a topic,” Reid points out.
To that end, Google is working to refine its detection mechanisms for nonsensical queries and improve the handling of satirical content, says Reid.
All told, Google maintains that user feedback has been positive, with higher satisfaction reported for search results featuring AI Overviews.
Users are “asking longer, more complex questions that they know Google can now help with,” Reid elaborates.
“They use AI Overviews as a jumping off point to visit web content, and we see that the clicks to webpages are higher quality—people are more likely to stay on that page, because we’ve done a better job of finding the right info and helpful webpages for them.”
She concludes: “At the scale of the web, with billions of queries coming in every day, there are bound to be some oddities and errors… We’ll keep improving when and how we show AI Overviews and strengthening our protections, including for edge cases, and we’re very grateful for the ongoing feedback.”
[via Washington Post and 9to5Google, images via various sources]
