AI Voice Simulator Falls Victim To Abusive Deepfakes Mimicking Famous People
As with everything else, there are two sides to the coin of deepfakes. On one hand, the technology recently allowed for A-listers to “star” in a comedy sketch program without actually having been at the ITV studios.
The other, a far more concerning one, involves the tool being used to target and hurt others, such as with non-consensual deepfake adult imagery. Or in this case, a voice generator being used to hurl abusive, homophobic slurs.
When UK-based ElevenLabs first released its Prime Voice AI earlier this month, it was expecting the text-to-voice generator to be used to emulate real-world voices that could be featured in audiobooks, video games, or even fan fiction.
According to Gizmodo, the mimicked voices sound relatively real, and a layperson may not be able to tell the difference between an actual clip of a celebrity speaking when compared to the fakes.
Those with sharper ears may still pick up on a “robotic quality” that appears in the sound, especially in longer clips.
Our current ideas:
— ElevenLabs (@elevenlabsio) January 30, 2023
(1) Additional account verifications to enable Voice Cloning: such as payment info or even full ID verification
(2) Verifying copyright to the voice by submitting sample with prompted text
(3) Drop Voice Lab altogether and manually verify each Cloning Request
As per a report by Motherboard, users from 4Chan used the system to deepfake voices of celebrities and popular influencers, such as Joe Rogan and Robin Williams, which on the surface doesn’t seem to be cause for alarm.
Alas, it doesn’t stop there. One user allegedly uploaded a clip of actress Emma Watson reading a portion of Adolf Hitler’s Mein Kampf, while another altered Justin Roiland’s voice from Rick & Morty to allude to domestic violence.
Gizmodo found other clips of the AI generator voicing “intense misogyny or transphobia,” using voices of characters from anime shows or video games, to which the company finally admitted it was noticing increasing “misuse cases.”
ElevenLabs has several options on the table to stifle the harmful content, including account verification—which would require users to pay upfront—or ending the free version of the algorithm altogether, allowing each request to be manually verified.
It’s undeniable governments and industries worldwide need to join hands in coming up with rules to regulate the use of AI systems, which although interesting and intelligent, have the capability to do harm when in the wrong hands.
[via Gizmodo and Motherboard, cover image via Andreysha | Dreamstime.com]
