Harnessing the Power of Vision and Language to Improve Surgical Safety
Prof. Nicolas Padoy
University of Strasbourg, France
报告时间：2025.04.03(周四） 16：00
报告地点：转化医学大楼E200会议室
Abstract: In this talk, I will introduce the concept of a Surgical Control Tower that leverages intra-operative data to detect, analyze, and support surgical activities in the operating room. I will highlight how artificial intelligence can improve surgical safety, focusing on timeout and "critical view of safety" maneuvers during digestive surgery. This will include a discussion of the clinical need, our formalization of the problem in a framework suitable for AI, and the use of modern computer vision techniques to enhance the performance of this critical step. Additionally, I will explore strategies for scaling AI adoption in surgery, emphasizing methods to reduce dependency on annotated data and expand to a broader range of clinical applications. In particular, I will present our efforts to develop a generalist vision-language model for surgery, trained using supervisory signals derived from surgical video lectures available on e-learning platforms. I will demonstrate how information extracted from these lectures, such as natural language transcriptions and visual content, can be used to create versatile representations, enabling zero-shot adaptation to various surgical tasks and procedures without fine-tuning.
Biography: Nicolas Padoy is a Professor of Computer Science at the University of Strasbourg, France, and the Scientific Director as well as Director of Computer Science and Artificial Intelligence Research at the IHU Strasbourg, a leading institute for minimally invasive surgery. He leads the CAMMA research group (Computational Analysis and Modeling of Medical Activities), which focuses on leveraging multimodal data from operating rooms with machine learning and computer vision to develop cognitive assistance systems and enhance human-machine collaboration. His research aims to improve the safety, quality, and efficiency of surgical procedures. Nicolas Padoy completed his PhD in 2010 jointly between the Technical University of Munich, Germany, and INRIA/University Henri Poincaré, France. Subsequently, he was a postdoctoral researcher and later an Assistant Research Professor in the Laboratory for Computational Interactions and Robotics at the Johns Hopkins University, USA. In 2015, Nicolas Padoy was awarded the Guy Ourisson Prize by the Cercle Gutenberg, which recognizes promising young researchers in Alsace. In 2020, he was awarded one of 43 national AI Chairs by the Agence Nationale de la Recherche (ANR) for his project AI4ORSafety and, in 2023, a prestigious European ERC Consolidator Grant for his project CompSURG. In recognition of his contributions to the field of surgical data science and of his engagement in the academic community, Nicolas Padoy was elected Fellow of the Medical Image Computing and Computer-assisted Intervention (MICCAI) society in 2024. He is currently General Chair of the international IPCAI conference on Information Processing in Computer-Assisted Interventions.
