When developing technologies to support surgeons in the operating room, it\u2019s vital to rigorously evaluate new AI models, workflow optimizations, and imaging techniques in digital simulations and laboratory studies before testing them in the real world.<\/p>\n
Led by John C. Malone Associate Professor of Computer Science\u00a0Mathias Unberath<\/a>, researchers from the\u00a0Advanced Robotics and Computationally AugmenteD Environments (ARCADE) Lab<\/a>\u00a0are advancing the science of surgery by using digital twins\u2014virtual replicas of a patient or operating room that are updated continuously with real-time data\u2014to refine new technologies without risking patient safety.<\/p>\n They have constructed\u00a0digital twins<\/a>\u00a0of operating rooms to support immersive space planning applications and\u00a0digital twins of patients<\/a>\u00a0as a means to minimize human exposure to harmful X-rays.<\/p>\n Now, at the\u00a028th International Conference on Medical Image Computing and Computer Assisted Intervention<\/a>\u00a0(MICCAI), they\u2019ve\u00a0demonstrated<\/a>\u00a0how OR digital twins can help machine learning models analyze surgical videos and identify workflow bottlenecks.<\/p>\n For an ML model to automatically identify potential cost savings, monitor safety protocol compliance, and evaluate team coordination in an OR, it must use reasoning segmentation, or RS, to interpret high-level user commands without explicit step-by-step guidance.<\/p>\n Traditional RS methods rely on large language model (LLM) fine-tuning, but even state-of-the-art LLMs struggle with complex medical terminology, data variability across hospitals and health care institutions, and identifying semantic and spatial relationships when analyzing OR videos.<\/p>\n \u201cConstant fine-tuning requires frequent updates to maintain compatibility with evolving models, potentially disrupting continuous monitoring and increasing implementation costs,\u201d the researchers write.<\/p>\n To address this issue, Unberath and his team\u2014graduate students\u00a0Yiqing Shen<\/a>, Chenjia Li, and Bohan Liu; alumnus Cheng-Yi \u201cCharlie\u201d Li, Engr \u201925 (MS); and Tito Porras, a resident in the\u00a0Department of Neurology and Neurosurgery<\/a>\u00a0at the\u00a0School of Medicine<\/a>\u2014propose a new approach that preserves the semantic and spatial relationships observed in OR videos while restructuring the traditional RS approach into a \u201creason-retrieve-synthesize\u201d paradigm, enabling adaptive RS without fine-tuning.<\/p>\n <\/p>\n