Nvidia Builds General Humanoid Operating System To Power Future Race Of Robots

Image via Nvidia

The world of The Jetsons is finally becoming a reality, with a colony of helpful Rosie the Robots upon us. Nvidia is developing a new artificial intelligence model called Project GR00T to make humanoid robots smarter and more efficient. GR00T—short for Generalist Robot 00 Technology—acts as a foundation model, essentially the AI brain for these robots, enabling them to acquire new skills and interact effectively with the real world.

Being multimodal, GR00T can process various inputs, including spoken instructions and past interactions. It then determines the most suitable actions for the robot to perform based on these insights.

Most notable of all is GR00T’s ability to learn from human demonstrations. By observing a few instances of a task, the robot can grasp essential skills like coordination and dexterity and mimic them. Its observations are additionally enhanced through Nvidia’s simulation environments, where robots can experiment and learn from mistakes in a safe, controlled setting. Additionally, GR00T can study video data containing human actions, further refining its motor skills and ability to emulate human movements.

To ensure smooth operation, this general-purpose operating system is paired with Nvidia’s Jetson Thor computer, a powerful AI system-on-a-chip specifically designed to run AI models such as GR00T. The combination provides the processing power needed for complex tasks.

Moreover, the project will utilize Nvdia’s Isaac robotics platform, which has received significant upgrades to support the development of robots powered by GR00T. This includes the introduction of Isaac Manipulator and Isaac Perceptor, offering state-of-the-art capabilities for robotic arms and multi-camera, 3D surround-vision systems. Such advancements are expected to improve efficiency, safety, and the overall autonomy of robots in manufacturing, fulfillment operations, and beyond.

“Building foundation models for general humanoid robots is one of the most exciting problems to solve in AI today,” shared Nvidia founder and CEO Jensen Huang. “The enabling technologies are coming together for leading roboticists around the world to take giant leaps towards artificial general robotics.”

Leading robotics companies like Boston Dynamics, 1X Technologies, and Agility Robotics are collaborating with Nvidia to leverage GR00T's capabilities. Their combined efforts will bring a force of robots that can perform tasks in numerous real-world applications, such as manufacturing and warehouse operations.

Similar to how we humans gain experience from all kinds of exposure, future robots will nurture wisdom and skills through different stimuli, adapting to our lives and assisting us in myriad ways.

[via IEEE Spectrum and Nvidia, cover image via Nvidia]