NVIDIA has unveiled Cosmos3-H-Dreams, a generative AI system that brings interactive surgical simulation to the visual and procedural complexity of clinical surgery. Built on the Cosmos 3 foundation model and accelerated with FlashDreams — NVIDIA's streaming-inference library — the system runs interactively on a single GPU, a significant step from its predecessor that was limited to tabletop suturing tasks.
Unlike traditional numerical simulators that explicitly model geometry, tissue deformation, and physical interactions, Cosmos3-H-Dreams learns visual dynamics directly from synchronized video and robot kinematics. The result is a world foundation model for surgery that generates realistic, responsive surgical scenes in real time.
The system was trained on clinical cholecystectomy (gallbladder removal) recordings from the CMR Surgical Versius platform — believed to be the first real-time generative surgical simulator trained on actual clinical procedures rather than simulated or tabletop data. Users can control surgical instruments (Cadiere grasper, scissors) and see the simulated tissue respond naturally to their inputs.
NVIDIA positions this as an alternative to expensive numerical simulation for surgical training, offering a more scalable approach that captures the visual richness of actual procedures. The system also supports knowledge distillation — a "four-step student" model that runs faster while maintaining quality — and streaming through FlashDreams for low-latency interaction.
