NVIDIA's Surgical Robot Passes the Simulation Test—and It's Heading to Hospitals in 2026

Source: NVIDIA | Published: 2026-08-11T13:00:34Z

NVIDIA's Cosmos surgical simulator achieves near-perfect alignment between simulated rankings and real-world performance, compressing weeks of lab validation into hours of simulation.

Three Foundations of Physical AI

Pera, head of NVIDIA's healthcare team, distilled two years of technical progress into three building blocks.

First, world foundation models. Cosmos was originally built to understand roads and traffic signs. NVIDIA fine-tuned it on surgical scenes and clinical workflows, teaching it the physics of the medical world.

Second, robot policy models. Groot H was officially released at GTC — fully open source, weights available on Hugging Face, code on GitHub. It takes visual perception and language understanding, converting them into robot action policies: the model that tells a robot what to do next.

Third, physical simulation. From classical simulation to neural simulation, surrogate simulation engines are approaching real-world fidelity. Together, these three foundations support the entire framework of medical physical AI.

Where 750 Hours of Data Comes From

Physical AI has a fundamental problem: surgical robotics is nothing like autonomous driving. You can't let a robot practice on real patients indefinitely.

Eighteen months ago, NVIDIA began building a dataset called Open-H. Pera singled out the founding members: Axel Krieger, Dr. Nassir Navab, Sean Huver, and Mahdi Azizian. What started as a conversation eventually brought together 35-plus partners, spanning 16 robotic form factors and accumulating over 750 hours of medical robot operation data — including real surgical footage contributed by leading companies like CMR Surgical and Moon Surgical.

This is where the entire data flywheel begins. Real data as the foundation, simulation engines to scale it up, and models that can generalize to scenarios they've never seen.

What Groot H Can and Can't Do

Groot H has a two-layer architecture: a Cosmos reasoning engine handles visual and language understanding to generate semantic tokens; a diffusion Transformer takes those tokens plus the robot's current state and outputs action commands.

Using vanilla Groot for surgical robotics doesn't work well. Groot H, fine-tuned on Open-H data, significantly outperforms zero-shot generalization from any general-purpose model in surgical settings.

But it's still not enough. Pera's assessment was blunt: pre-training plus real-data fine-tuning still fails to cover all the edge cases that arise in surgery. That's why synthetic data generation and simulation loops are non-negotiable parts of the workflow — not nice-to-haves.

Practicing in Simulation to Reduce Trial and Error on Patients

For this, NVIDIA released three components under Cosmos-H-Surgical.

Cosmos-H-Surgical-Transfer handles data augmentation: it constructs anatomical models from CT/MR scans, builds digital twin environments in Isaac Sim, and uses Cosmos to generate variations while maintaining physical consistency. Cosmos-H-Surgical-Predict lets models anticipate different action states and practice multiple skills within a single environment. Cosmos-H-Surgical-Simulator is a learnable simulator where robot policies can be evaluated repeatedly — Pera cited the team's published research: rankings of Groot, Pi Zero, and other models inside this Cosmos simulator correlate closely with their rankings in the real world.

The implication is significant: validating a policy used to take days or weeks in the lab. With the simulator, you might get an answer in hours.

From the Operating Room to the Hospital Corridor

At GTC, NVIDIA extended Isaac for Healthcare beyond surgical robotics.

Rheo is a development blueprint for hospital automation: how to build a digital twin, kick off a data flywheel, train and test policies, and deploy to the real environment. Peritas AI's humanoid robot DexMate is already using this workflow, with Advent Health as the partner hospital and the pilot running in real corridors outside actual operating rooms.

Pera's call: 2026 will be the year these robots move into deployment.

The Deployment Stack: Holoscan 4.0

Holoscan 4.0 was announced at GTC as the deployment infrastructure for physical AI.

Three key features: direct EtherCAT control of robot motors; interoperability with existing development stacks like ROS 2; and GPU-resident compute graphs that bypass the CPU entirely, driving latency as low as possible.

Paired with IGX Thor — NVIDIA's industrial-grade edge AI platform with 8x the compute of its predecessor — the logic is straightforward: the models running on robots in the future won't be simple detection networks. They'll be VLMs and VLAs. Edge hardware decisions need to happen now, or those models simply won't run later.

Reading Physical Insights Directly from Raw Signals

GTC also introduced a model targeting a different challenge: Raw-to-Insights.

The traditional workflow is: raw sensor signal → image reconstruction → AI analyzes the image. NVIDIA wants to skip the middle step and have AI learn directly from raw RF signals.

The ultrasound version was developed in collaboration with the Siemens research team, built on the IGX and HSB platforms. It performs real-time patient-adaptive imaging and has reached production-ready status. The MRI version ranked first at the CMRX Reconstruction Challenge, replacing CUDA-based classical reconstruction pipelines with AI.

How much information quietly disappears during traditional image reconstruction? There's no definitive answer yet. Pera characterized the research as still early, but called it one of the most important topics to watch at academic conferences over the next few years.

How Much Road Remains from Simulation to Reality

During the post-talk Q&A, someone asked directly: when going from simulation to real deployment, what's the first thing to break?

Pera didn't dodge it: "Too many things break — it's hard to say which goes first. Physical consistency of the digital twin, port connections, system-level issues — this is not a seamless transition."

His broader view: the debate between pure simulation training versus pure real-data training is a false dichotomy. The future is hybrid. Certain physical details of tool-tissue interaction must rely on classical simulation. But for rapidly generating variations and covering edge cases, neural simulation offers cost advantages by an order of magnitude. NVIDIA's goal isn't to replace companies like Surgical Sciences that specialize in classical simulation — it's to work alongside them to drive down the cost and time of building high-fidelity surgical simulators, making the toolchain accessible to a wider range of surgical procedures.