YouTube Announces AI-Powered Dubbing Tool To Easily Translate Creators’ Videos

At the recent VidCon, YouTube announced it is currently testing a new artificial intelligence tool that would help creators automatically translate their videos to be dubbed in other languages. The streaming giant has partnered with Google’s in-house incubator service Aloud on the project.

This step to introduce multi-language dubbing follows the site’s decision earlier this year to include support for audio tracks, which allowed creators to add dubs in different languages to new or existing videos to cater to a wider audience internationally.

According to TechCrunch, as of June 2023, creators have dubbed over 10,000 videos in more than 70 languages. The new addition from Aloud will allow even more clips to be dubbed at no additional cost instead of having to outsource the job to costly third-party providers.

The AI-powered tool works by transcribing the video in its original language before translating the content and producing a dubbed version of the clip. Helpfully, creators will be able to review and edit the transcription before different dubs are generated to correct any mistakes.

At the moment, YouTube is testing out the feature with hundreds of creators on its site before opening it to all users. While Aloud can only dub videos in English, Spanish, and Portuguese currently, more languages, including Hindi and Bahasa Indonesia, will be available in the future.

Interestingly, Amjad Hanifl, vice-president of creator products, said the site is working on a more advanced version of the technology that would allow dubs to sound like creators’ voices instead of the typical monotone, robotic drone.

Going forward, more features could be added to Aloud, including the ability to preserve a creator’s voice, better transfer of emotions, and even reanimating the lip movements of talking heads. Though, it remains to be seen if these feats can be accomplished in the near future.

[via Dexerto and TechCrunch, cover image via Nikkimeel | Dreamstime.com]