Lips Can Now Be Deepfaked To Mimic Dubbed Translations In Films

By Mikelle Leow, 07 May 2021

Image via Paramount Pictures / IMDb

It’s 2021, and despite advancements in AI, the experience of watching a foreign film still doesn’t feel so natural. Movie buffs might argue that switching on subtitles prevents them from enjoying the full artistic nature of a feature, and the other alternative—besides learning a whole new language—is to make do with watching unsynchronized lip movements in a dubbed version.

If filmmakers are listening, all this could change. A London tech startup, called ‘Flawless’, has built an AI that can seamlessly superimpose an actor’s lip movements and facial muscles to go in sync with translations. The company demonstrated the technology in a few videos, which depict Hollywood hotshots Tom Hanks and Robert De Niro realistically “speaking” in Japanese, Spanish, and German.

Fears about technology of this ilk are justifiable, as it makes it easier to manipulate speech of important figures and fool others into thinking they had said those things. When placed in the wrong hands, this sort of AI has the potential to turn into political warfare.

For now, Flawless’ AI seems to work on well intentions, and it does a pretty great job too. It aims to capture “all the nuance and emotions of the original material” so viewers can enjoy the intended experience of films.