Medical Augmented Reality

See TUMonline

In-situ visualization in medical augmented reality (AR) enables the registered view of virtual data such as a 3D CT scan, with a video view of the real anatomy of a patient. Data can be aligned with the required accuracy such that surgeons do not have to analyze data on an external monitor in the operating room. Instead, surgeons get a direct view onto and into the patient. Mental registration of medical imagery with the operation site is not necessary anymore. Augmenting medical images and virtual surgical instruments within the body provides the most intuitive way to understand the patient’s anatomy within the region of interest. This allows for the development of completely new generations of surgical navigation systems. Our Chair for Computer Aided Medical Procedures (CAMP) is one of the strongest and leading research centers for medical AR world-wide, featuring the world’s most accurate video-see-through AR system based on a head-mounted-display. Also, CAMP has developed the world’s first AR-enabled C-arm, CamC, which has already been tested in clinical trials in Munich. The objectives of this lecture are to have students learn the theoretical basics and practical aspects of augmented reality solutions related to the medical field. The content of the lectures are outlined below.

Lecture: + Covering the topics of Augmented Reality (AR) in the medical context + Basics of intra-operative imaging and navigation + Camera calibration using multi-view approaches + Tracking technologies (RGB, infrared, marker-based, markerless) + Projective geometry basics including transformations and reconstruction + Review of multimodal registration, visualization and user interfaces + Basics of rendering, avatars, and sonified interaction (depending on the time) + Includes real examples using head-mounted display (HMDs), camera augmented mobile C-arm (CamC), and AR magic mirror for anatomy education or alike (depending on the possibilities of the COVID situation) + Includes visits to the Navigated Augmented Reality Visualization Systems Laboratory, Chirurgische Klinik und Poliklinik Innenstadt; and IFL: Interdisziplinäres Forschungslabor, Klinikum rechts der Isar (depending on the possibilities of the COVID Situation, these will be in small groups or canceled) Exercises (project oriented/every two weeks/to be announced regarding the COVID context): + Using Python, C#, or C++ + Implementation of lecture course topics and examples Term-related information is provided in the Moodle course page.

Students should have robust programming skills and should be interested in Augmented- and Virtual Reality technologies in the medical context. Previous knowledge in Unity3D or UnrealEngine is usually of benefit, but not required.

Terry Peters and Kevin Cleary - Image Guided Interventions - Technology and Application – Springer 2008 - ISBN-10: 0387738568 Workshop and Proceedings of the following congresses - IPCAI/CARS - MICCAI - SMIT - ISMAR