NVIDIA Showcases How Developers Turn Ideas Into Simulations With Frontier AI Agents and Omniverse

The core claim of the official NVIDIA blog is not complicated: in the past, turning a simulation idea into a runnable application required manually assembling assets, connecting physics and rendering, and then checking scene behavior item by item; now developers can hand this work to frontier AI agents, leaving themselves responsible only for giving natural language instructions, reviewing results, and guiding revisions. The blog defines the role of Omniverse libraries as providing GPU-accelerated physics, rendering, and sensor simulation capabilities, while the agents are responsible for stringing these capabilities together.

This throughline appears repeatedly across the six cases, but each case "strings things together" differently: some have the agent connect four types of libraries—physics, scene updates, rendering, and interface; some have the agent first draw out a workflow and then integrate it in stages; some have the agent measure differences and directly modify OpenUSD scenes. Understanding these differences is more informative than remembering the conclusion that "agents can do simulation."

NVIDIA Omniverse product manager Frank DeLise used Astra to turn a SimReady warehouse and a humanoid robot into an interactive simulator with first- and third-person views. According to the blog, he directed Astra to connect Omniverse libraries including physics (ovphysx), scene updates (ovstage), rendering (ovrtx), and interface (ovui), and used SimReady (simready-foundation) to create the physics scene, with Astra generating animation and application code.

The value of this case is that it shows a complete slice of the agent workflow: asset preparation, physics connection, rendering, interface, animation, and code generation are placed in the same instruction chain. But the blog does not say how this simulator performs on warehouse tasks, nor does it give any performance numbers.

NVIDIA simulation technology team manager Doyub Kim had Astra build a reusable simulation environment based on San Francisco's Market Street, called Zero to Alpamayo. According to the blog, Kim first had Astra draw out the workflow, then connected asset creation, traffic, Omniverse RTX sensor simulation, and Alpamayo driving in stages, checking the integration item by item. This prototype was used as a testbed for comparing models and tracking how scene or sensor changes affect downstream driving behavior.

Another Cosmos3-Nano experiment changed the weather and lighting in recorded simulation videos, to compare how driving models react under different conditions in the same scene. The blog does not give the conclusion or scope of this experiment, so for now it can only be read as a demonstration direction, not a verified conclusion.

Ashley Reid, who is responsible for RTX sensor validation, directed Astra and a Claude Fable 5 agent to compare ovrtx camera and raw LiDAR output against recorded data, creating two digital twins from scratch and improving two existing digital twins. The blog says Reid guided the iterative process in about three days, with the agent measuring differences, creating or modifying OpenUSD scenes, and checking results, and acceptance depending on camera and LiDAR metrics.

This is the only one of the six cases that gives both a time figure (about three days) and a quantity figure (2 created from scratch, 2 improved), and the only one that explicitly states acceptance criteria. But the blog does not disclose the specific thresholds for the camera and LiDAR metrics, so the strictness of "acceptance passed" cannot be judged.

Tae Kim, who is responsible for Omniverse engineering and product, used sports videos and natural language instructions to guide Astra in building Robo Olympics, testing a simulated Unitree G1 humanoid robot doing sports. According to the blog, Astra built controllers under Kim's guidance and improved them through physics trials; the Newton Physics Engine simulated behavior, the open-source NVIDIA Warp framework accelerated computation, and ovrtx rendered the scene and virtual camera images.

In one experiment, the robot cleared a single hurdle in 64 of 100 simulation trials. This is the only success-rate figure given in the entire blog, but it corresponds only to the specific experimental condition of "a single hurdle"; the blog does not explain reproducibility or statistical methodology, nor does it say whether this ratio represents a stable level for the model or controller. Reading it as "humanoid robot locomotion capability reaches 64%" is overinterpretation.

Jens Jebens, senior product manager for OpenUSD, directed Astra to model a car suspension in PTC Onshape and configure it in NVIDIA Isaac Sim; the agent measured the available space and designed a wrench able to reach the bolt, and Jebens reported successfully removing the suspension component in simulation. This case connects design decisions with disassembly results, but "successfully removed" is self-reported by the employee involved, with no external review.

NVIDIA engineering director Nic Johns used a single prompt to have Astra assemble NASA assets into an OpenUSD International Space Station model with telemetry, and used follow-up prompts to move the scene to the daylit side of Earth. Chirag Majithia of the Isaac engineering applications team directed Astra to turn stereo camera footage into an editable OpenUSD studio, with the process combining PyCuSFM, FoundationStereo, and nvblox, and using USD Content Agents to configure object motion and interaction. The two cases use different verification methods: the former advances through prompts and corrections, while the latter relies on Isaac Sim tests to guide collision and contact revisions.

All cases come from the official NVIDIA blog and are vendor self-reports, lacking independent third-party verification. The blog does not disclose resource conditions such as compute, cost, or time for each case, nor does it say whether these examples have been used in actual product development or are merely demonstrations.

The agent workflows depend on NVIDIA Omniverse libraries, Isaac Sim, OpenUSD, and specific frontier models (GPT-6 Astra, Claude Fable 5). The blog does not explain the specific capability boundaries and version information of GPT-6 Astra, nor does it explain the reliability of natural language-directed agents at larger scale or in production environments. Therefore, what these cases can prove is that "within NVIDIA, led by employees familiar with the toolchain, under specific conditions, it can run through"—not that "this workflow can already be reliably replicated by any team."