Google’s New Chatbot Freaks People Out With Its Near-Human Conversation Skills
Images via Shutterstock and Wikimedia Commons
Google’s latest chatbot is leaving internet users tongue-tied with its uncanny charisma and excruciating dad jokes.
On Tuesday, the tech giant introduced Meena, a bot it says has more sophisticated conversational skills “than existing state-of-the-art chatbots.” As lofty as the claim seems, Google made sure to test Meena’s accuracy with a metric it calls the ‘Sensibleness and Specificity Average’ (SSA).
To test the chatbot’s precision, Google had human moderators start 100 freeform discussions with Meena and various open-domain chatbots.
Every time the bots replied, the human participants evaluated their communicational abilities with the SSA model. First, they had to appraise the bots’ replies based on whether they made logical and contextual sense. If they were, moderators delved deeper and assessed if the responses were “specific” to the conversation, rather than generic.
For instance, if a worker said, “I love tennis,” and Meena replied, “That’s nice,” the response would be labeled as “not specific.” However, if Meena answered, “Me too, I can’t get enough of Roger Federer!” the message would fall under “specific.”
On Twitter, social media users are flabbergasted at Meena’s remarkable conversational skills after a number of transcripts went around.
A couple of them show Meena telling jokes that go on even after several text exchanges.
In one example, the bot even spit wisecracks that could not be found in the database.
If you’re wondering how close Meena’s conversational skills are with a human’s, Google revealed in a blog post that they’re almost one-to-one. On average, a human would get an SSA score of 86 percent, while Meena attained an incredible 79 percent. In contrast, other chatbots scored between 31 and 56 percent.
um.. google's latest chatbot is 😳 https://t.co/jJyW8pMeMZ pic.twitter.com/XpOkp0Pd5E
— Kyle McDonald (@kcimc) January 29, 2020
Right before it decides to kill you and steal your body for itself. pic.twitter.com/BLGX6MGKYC
— David Liu (@thedak) January 29, 2020
My new bff is a chatbot https://t.co/2d2JVrjjkG
— Perrin Drumm (@perrindrumm) January 29, 2020
and here i thought i had an industrialization-proof job :( https://t.co/sXzkJPaj01
— Riff Conner (@rifflesby) January 29, 2020
[via Futurism, images via various sources]
Also check out these recent news
