Apple Shows Off ‘GAUDI’ Architect Bot Tool That Turns Text Into 3D Scenes

Apple’s newest architect practices with the same building blocks as OpenAI’s DALL-E, Google’s Imagen, and Midjourney. In what could lay out the bricks for its rumored mixed-reality headset, the tech giant has unveiled ‘GAUDI’, a neural architecture tool that generates 3D scenes.

Named after famed Spanish architect Antoni Gaudi, GAUDI differs from other programs in that it can create immersive, moving 3D scenes in various camera perspectives. Tools like DALL-E are only trained to generate 2D artworks, which would then serve as backdrops to be rendered further into 3D.

There are notable limitations to imaging multiple perspectives from 2D objects. Especially with architectural scenes, the AI may literally run into a wall or an obstacle, diminishing the ability to project every possible camera position around the object.

As demonstrated by Miguel Angel Bautista, a scientist at Apple’s machine-learning research team, GAUDI is able to render 3D indoor scenes from moving cameras or text prompts, the latter of which include simple phrases like “go through the hallway” or “go down the stairs.” It can also start from regular images and imagine them in 3D.

Excited for this to be out! Introducing GAUDI: a generative model for 3D indoor scenes. We tackle the problem of learning a generative model of 3D scenes parametrized as radiance fields. This has been a great collaboration across multiple teams at @Apple. https://t.co/aJOqtzA2CI https://t.co/tSkJdXK31C pic.twitter.com/ReeXAPGg95

— Miguel Angel Bautista (@itsbautistam) July 29, 2022

Several Apple teams got together to work on the complex technology, and its research has been detailed in a new paper.

For now, GAUDI’s results are pretty low-quality, but it does offer a snapshot of what Apple might do to improve on its LiDAR technology and build into its future mixed-reality products.

[via Patently Apple and Mixed News, video and cover image via Miguel Angel Bautista]