Italy May Soon Shut Down AI ‘Girlfriends’ Over Safety Concerns

Those in Italy seeking love or friendship on artificial intelligence chatbot Replika could soon have to look elsewhere, as the country recently ordered the San Francisco-based firm to halt processing its citizens’ data due to safety fears.

Replika, which has been in operation since 2017, allows users to create custom avatars that can interact with them, and has claimed to be a method of improving one’s well-being. More notably, it promotes adult messaging as one of its biggest draws for paid subscriptions.

According to PC Mag, Italy’s Data Protection Agency felt the application’s ability to influence users’ moods “may increase the risk for individuals still in a developmental stage or in a state of emotional fragility.”

Simply put, the country is worried these AI “friends” or “girlfriends” could pose a threat to minors or those who are struggling emotionally. In addition, the agency said the developer failed to comply with its legal requirements for using customers’ data, especially in the case of children.

In a press release, authorities explained that Replika poses a risk to children as it offers replies that are “absolutely inappropriate to their age,” a concern made worse by the application’s lack of an age-verification system.

There are currently still no measures to block underaged users from accessing the application, which serves as the key issue. Going forward, as per Futurism, the firm has 20 days to comply with Italy’s rules, or it could be fined up to US$21.5 million.

Will more countries begin to clamp down on AI chatbots or put protections in place to ensure these systems are only accessible to adults? Given how quickly the space is expanding, it’s certainly pertinent for authorities worldwide to address the growing issue.

[via PC Mag and Futurism, cover image via 純一 島å´Å½ | Dreamstime.com]