At SIGGRAPH, NVIDIA announced new Omniverse libraries and Cosmos AI models to speed up robotics development. These tools let developers create digital twins, simulate the real world, and generate synthetic data for physical AI models. Using NVIDIA RTX PRO Servers and DGX Cloud, developers can run demanding simulations from anywhere, enabling faster innovation.
Omniverse Libraries and Robotics Simulation
NVIDIA’s Omniverse SDKs and libraries improve data interoperability between MuJoCo (MJCF) and Universal Scene Description (OpenUSD). This update helps 250,000 robot learning developers simulate robots across different platforms. Additionally, Omniverse NuRec libraries enable RTX ray-traced 3D Gaussian splatting to capture and simulate the real world using sensor data.
Moreover, the new versions of Isaac Sim™ 5.0 and Isaac Lab 2.2 are now available on GitHub. They include updated robot and sensor schemas, helping developers close the simulation-to-reality gap.
Cosmos AI Models for World Generation
NVIDIA’s Cosmos models, downloaded over 2 million times, help developers generate synthetic data for training robots. Cosmos Transfer-2 accelerates photorealistic synthetic data creation from 3D simulation scenes. The distilled version simplifies the 70-step process, letting developers run models on RTX PRO Servers faster than ever before.
As a result, Lightwheel and Skild AI are using Cosmos Transfer to simulate complex conditions and scale their AI training effectively.
Cosmos Reason: Advancing World Understanding
Cosmos Reason is a 7-billion-parameter model that enables robots to reason with prior knowledge, physics, and common sense. This model enhances various tasks:
- Data curation and annotation, automating training dataset creation.
- Robot planning and reasoning, supporting deliberate decision-making.
- Video analytics, helping AI agents analyze video data for root-cause analysis.
For example, Uber and Magna are using Cosmos Reason to annotate autonomous vehicle data, improving their adaptability to new cities and environments.
New NVIDIA AI Infrastructure for Robotics
NVIDIA introduced RTX PRO Blackwell Servers and DGX Cloud to handle demanding robotics workloads. These systems streamline training, synthetic data generation, and simulation. Additionally, DGX Cloud, available on Microsoft Azure Marketplace, offers a managed platform to simplify streaming Omniverse and NVIDIA RTX-based applications, reducing infrastructure management.
Accelerating Robotics and AI Development
NVIDIA also promotes 3D simulation technologies through the OpenUSD Curriculum and Certification program. Supported by industry leaders, this initiative meets the growing demand for USD expertise. In addition, NVIDIA collaborates with Lightwheel to integrate robot policy training and evaluation frameworks into Isaac Lab, enhancing robot learning capabilities.
In conclusion, NVIDIA continues to lead robotics development, empowering developers to create smarter, more efficient AI systems.
