Workflow-Anchored Memory
WAM writes timestamped Observe records at stable instrument–action–target changes. At each confirmed phase boundary, it replaces that phase’s records with a compact Summary, retaining the rest of memory.

A Workflow-Anchored Proactive Streaming Framework
and Benchmark for Real-Time Surgical Video Understanding
✉ Corresponding authors
The project website and research manuscript are now available.
arXiv and publication updates will appear here when available.
Jointly perceive the ongoing scene, remember past surgical events, and respond before clinically relevant moments pass.
Explore the recorded examplesSurgStream combines workflow-structured memory with response supervision within valid procedural windows.
WAM writes timestamped Observe records at stable instrument–action–target changes. At each confirmed phase boundary, it replaces that phase’s records with a compact Summary, retaining the rest of memory.
DTL redraws each target response time inside its evidence-to-deadline window in every training epoch, favouring earlier valid times. Sparse state supervision teaches when to wait and when to respond.
Explore five temporal modes. Replay the source footage and follow when evidence becomes available and a response is recorded.
Answer questions about the visible surgical scene as frames arrive.
Selected recorded research outputs, not an accuracy estimate. Each website video excerpt is continuous; timestamps refer to the original recordings. The main film shows the FP request as pending from an earlier time. CM summaries are model-generated.
Example records11,667 streaming interactions across various tasks evaluate what a model says and when it responds. Expert annotations ground the questions, evidence times and deadlines.
Built from Cholec80 and CholecT50, with shared recordings grouped by procedure to prevent split leakage.
Code and benchmark release links will be added here.