TikTokers Not Just Influencing Fans But Alibaba’s ‘Animate Anyone’ AI Too

Photo 206409660 © Vadim Arzyukov | Dreamstime.com

What do Alibaba and TikTok have in common? They’re both not only the tech offspring of Chinese parents but are also powered, in part, by dancing influencers.

Alibaba has recently shown off its Animate Anyone model, developed by an in-house research team, that poses the capability to transform static images into dynamic dance videos, though it’s not without its controversies.

Animate Anyone employs a novel framework called ReferenceNet, integrating diffusion models to merge detailed features through spatial attention. It’s designed to be user-friendly, requiring no digital editing expertise, thus opening the door to quality animations for a broader audience.

The technology behind Animate Anyone stems from academic research and has been commercialized by Alibaba. The results are reportedly superior to previous attempts in this field, with the AI-generated dance sequences closely resembling the original. The model can also transform photos into videos, controlled by desired pose sequences, achieving temporal continuity with minimal glitches.

But, as it turns out, the artificial intelligence system was also trained using a dataset scraped from TikTok videos, featuring a wide array of users, including some of the social network’s most popular creators. High-profile names like Charli D’Amelio, Addison Rae, Ashley Nocera, Stina Kayy, Jasmine Chiswell, Mackenzie Ziegler, and Anna Sulcova were among those whose content was used, presumably without their consent, per 404 Media.

The model takes an input, such as a TikTok dance video, and outputs a new version, allowing variations in clothing or styles.

RIP TikTokers 💀 pic.twitter.com/DG5y13Dn5l

— Dreaming Tulpa 🥓👑 (@dreamingtulpa) December 2, 2023

In a demonstration, Alibaba used the app to animate the Terracotta Warriors, making them dance to a viral challenge. This showcased the model’s ability to animate full-body photos into dance videos while preserving original facial expressions and backgrounds.

The team behind Animate Anyone plans to make the demo and code publicly available. 

And while its makers may justify that TikTok content falls under fair use, critics question the ethics of this duet.

[via 404 Media, Decrypt, Technode, images via various sources]