Google Introduces Video Generator That Creates Surreal Clips From Little Input
There’ll come a time when, instead of sourcing the right props, training animals to perform tricks, or applying CGI techniques, you’ll simply have to tweak a few keywords and hit ‘Enter’.
Researchers from Google Research and the Hebrew University of Jerusalem have created ‘Dreamix’, a program that can use whatever source material you have to dream up videos matching your vision. It’s the latest of a wave of generative AI tools that require nothing more than text prompts to create art in an instant. Just recently, Google showed off a machine-learning model that can produce tracks of any genre from simple text.
Expanding beyond text prompts, Dreamix also accepts reference photos or videos to better fit what users have in mind. It boasts three key functions: video editing, image-to-video generation, and subject-driven generation.
In the case of video editing, Dreamix can easily recreate input videos and replace things with your preferred objects. For instance, you can upload a clip of a person cutting into a fruit, and then have the AI swap out the fruit for a slice of cake. Give it some footage of a field and it can throw in “fire on all sides,” if that’s what you want.
Video via Dreamix (CC BY-SA 4.0)
Video via Dreamix (CC BY-SA 4.0)
AI’s idea of cats is a tad creepy, but it’s not a bad effort when you consider that these kitties were originally puppies.
Video via Dreamix (CC BY-SA 4.0)
No animals were harmed in the making of this stunt video of a deer skateboarding in the wild.
Video via Dreamix (CC BY-SA 4.0)
Dreamix can perform image-to-video tasks, in which it reimagines single images and animates them.
Video via Dreamix (CC BY-SA 4.0)
Video via Dreamix (CC BY-SA 4.0)
This video of Bigfoot is just as blurry as recorded, alleged sightings of Sasquatch.
Video via Dreamix (CC BY-SA 4.0)
Video via Dreamix (CC BY-SA 4.0)
Video via Dreamix (CC BY-SA 4.0)
Video via Dreamix (CC BY-SA 4.0)
Finally, Dreamix can take a set of reference photos—or a mood board, if you will—combine it with a text prompt, and then bring it all to life.
Ted, who? This toy bear needs just a little push to become sentient.
Video via Dreamix (CC BY-SA 4.0)
Video via Dreamix (CC BY-SA 4.0)
Toy Story, but make it instant.
Video via Dreamix (CC BY-SA 4.0)
The technology could be a game-changer for creatives, who don’t necessarily think in verbal language and might instead be fueled by visual inspiration.
As Gadget Tendency points out, the results are fuzzy at best. However, the tool proves that AI understands context.
Dreamix won’t deliver perfect images, but it could be useful for accelerating the storyboarding process or generating reference illustrations.
For now, Dreamix’s resources aren’t open to the public. Although, it will only be time before others develop similar AI-assisted video editors that will potentially shake up the animation and filmmaking industries.
[via Dreamix, bleedingedge.ai, Gadget Tendency, videos and cover image via Dreamix (CC BY-SA 4.0)]
