Skip to content
Technical University of Munich

Medical Augmented Reality

Medical Augmented Reality (IN2293)

Lecturer (assistant)
Nassir Navab [L]
Nassir Navab [L]
Ulrich Eck
Tianyu Song
Michael Sommersperger
Sasan Matinfar
Maximilian Fehrentz
Laura Schütz
Number 0000000873
Type lecture with integrated exercises
Duration 4 SWS
Term Wintersemester 2025/26
Language of instruction English
Position within curricula See TUMonline
Dates See TUMonline

Admission information

See TUMonline

Objectives

In-situ visualization in medical augmented reality (AR) enables the registered view of virtual data such as a 3D CT scan, with a video view of the real anatomy of a patient. Data can be aligned with the required accuracy such that surgeons do not have to analyze data on an external monitor in the operating room. Instead, surgeons get a direct view onto and into the patient. Mental registration of medical imagery with the operation site is not necessary anymore. Augmenting medical images and virtual surgical instruments within the body provides the most intuitive way to understand the patient’s anatomy within the region of interest. This allows for the development of completely new generations of surgical navigation systems. The objectives of this lecture are to have students learn the theoretical basics and practical aspects of augmented reality solutions related to the medical field. The content of the lectures are outlined below.

Description

Lecture:
+ Basics of intra-operative imaging and navigation.
+ Camera calibration using multi-view approaches.
+ Projective geometry basics including transformations and reconstruction.
+ Introduction to XR/AR display technologies
+ 3D user interfaces and interaction methods
+ Basics of user-centered design with a focus on XR/AR
+ Different sensor technologies including: RG, infrared cameras, optical trackers, and depth-cameras.
+ Advanced perceptual visualization methods for in-situ augmentation
+ Review of multimodal registration, visualization and user interfaces.
+ Real-examples using head mounted display (HMDs), Camera augmented mobile C-arm (CamC) and, intraoperative gamma probes coupled with AR capabilities, AR magic mirror for anatomy education.
+ Includes visits to the Navigated Augmented Reality Visualization Systems Laboratory, Chirurgische Klinik und Poliklinik Innenstadt; and IFL: Interdisziplinäres Forschungslabor, Klinikum rechts der Isar.

Exercises (every two weeks):
+ Using Unity3D, C#, Python or C++
+ Implementation of lecture course topics with students in small groups.

Prerequisites

Students should have robust programming skills and should be interested in Augmented- and Virtual Reality technologies in the medical context. Previous knowledge in Unity3D or UnrealEngine is usually of benefit, but not required.

MA0901 Linear Algebra for Informatics
IN0038 Real-time Computer Graphics

Teaching and learning methods

Lectures and Exercises

Examination

Written Exam

Recommended literature

Terry Peters and Kevin Cleary - Image Guided Interventions - Technology and Application – Springer 2008 - ISBN-10: 0387738568

Workshop and Proceedings of the following congresses
- IPCAI/CARS
- MICCAI
- SMIT
- ISMAR

Links

To top