Abstract
The recent emergence of surgical data science holds the promise to enable a new generation of operating room (OR) support systems. The fine-grained localization of clinicians in the OR, either at the keypoint-level using human pose estimation or at the pixel-level using instance segmentation, is a key component to design such systems.
In this dissertation, we explore unsupervised domain adaptation (UDA) methods to enable visual learning for the target domain, the operating room, by working in two complementary directions. First, we study how low-resolution images with a downsampling factor as low as 12x can be used for fine-grained clinicians localization to address privacy concerns. Second, we propose several self-supervised methods to transfer learned information from a labeled source domain to an unlabeled target domain to address the visual domain shift and the lack of annotations. These methods employ self-supervised predictions in allowing the model to learn and adapt to the unlabeled target domain.
To demonstrate the effectiveness of our proposed approaches, we release the first public dataset, called the multi-view operating room MVOR, generated from recordings of real clinical interventions. We obtain state-of-the-art results on the MVOR dataset, specifically on the privacy-preserving low-resolution OR images. We hope our proposed UDA approaches could help to scale up and deploy novel AI assistance applications for the OR environments.
Jury members
- Prof. Nicolas Padoy, University of Strasbourg, IHU Strasbourg, France
- Prof. Afshin Gangi, CHU Strasbourg, France
- Prof. Gregory Hager, Johns Hopkins University, USA
- Prof. Michel de Mathelin, University of Strasbourg, France
- Prof. Slobodan Ilic, Technical University of Munich (TUM), Germany
- Dr. Vasileios Belagiannis, University of Ulm, Germany