SonoWorld: Towards Interactive Pose-Controllable Ultrasound World Models

Anonymized Authors
Under Review
Paper (Coming Soon) GitHub (Coming Soon)
SonoWorld rollouts: three 6-DoF probe paths from a shared initial frame, generated ultrasound videos, and reconstructed 3D thyroids

From a single shared initial frame, SonoWorld generates future ultrasound observations conditioned on different EM-tracked 6-DoF probe trajectories (Paths A–C). Each rollout stays anatomically consistent enough to reconstruct a 3D thyroid volume from the generated frames.

Abstract

Robotic ultrasound can standardize image acquisition and reduce operator workload and variability, but policy learning remains limited by the lack of realistic, controllable simulation environments: physical phantoms and acoustic simulators exhibit a substantial domain gap from clinical images, and trial-and-error exploration on patients is unsafe. While controllable world models have advanced rapidly in natural-image domains, ultrasound world modeling remains challenging because each frame captures only a cross-section of unobserved 3D anatomy, causing small probe motions to induce large appearance changes. Existing approaches rely on robot-tracked poses and latent-space prediction, but do not directly generate the observations required for interactive control.

We propose SonoWorld, a causal ultrasound world model that learns video dynamics from unlabeled clinical videos and grounds them to 6-DoF control using a small set of freehand clinical scans tracked with an electromagnetic (EM) sensor. SonoWorld is trained in three stages: pretraining a latent-action-conditioned video diffusion model on around 6.7 million unlabeled multi-organ ultrasound frames; grounding the learned video dynamics to metric probe poses with a small EM-tracked thyroid dataset; and applying distribution matching distillation to enable causal, long-horizon autoregressive generation with selectively caching eviction tokens at inference time.

We evaluate controllability through pose-following fidelity, round-trip consistency, and axis-specific control under synthetic actions on held-out tracked scans, and further validate it with a user study. In model-based action discrimination, SonoWorld reaches thyroid target planes with a success rate above 74%. To our knowledge, SonoWorld is the first causal, pose-controllable ultrasound world model learned primarily from freehand clinical videos, providing a data-driven interactive environment for robotic ultrasound data acquisition and policy evaluation.

Method Overview

SonoWorld is trained in three stages:

  • Latent-Action Pretraining — a latent-action-conditioned video diffusion model learns ultrasound dynamics from ~6.7M unlabeled multi-organ clinical frames
  • Pose Grounding — the learned dynamics are grounded to metric 6-DoF probe poses using a small EM-tracked freehand thyroid dataset
  • Causal Distillation — distribution matching distillation enables causal, long-horizon autoregressive generation, with selective eviction-token caching at inference

Results

Controllability is evaluated on held-out EM-tracked scans:

  • Pose-Following Fidelity — generated views track commanded probe trajectories
  • Round-Trip Consistency — returning to a pose recovers the same anatomy
  • Axis-Specific Control — responses to synthetic actions along each DoF
  • Model-Based Action Discrimination: > 74% success reaching thyroid target planes
  • User Study — interactive validation of controllability

BibTeX

@inproceedings{sonoworld2027,
  title={SonoWorld: Towards Interactive Pose-Controllable Ultrasound World Models},
  author={Anonymous},
  year={2027}
}