CAMP Sonification Team
Auditory Intelligence for Interactive Medicine
The CAMP Sonification Team investigates Auditory Intelligence for Interactive Medicine, developing next-generation computational methods that transform physical interactions, medical imaging, and AI models into informative auditory representations. As modern medical procedures increasingly rely on imaging, robotics, and artificial intelligence, clinicians face growing visual complexity and cognitive demands. Our research explores how sound can become a complementary information channel that augments perception, enhances situational awareness, and supports decision-making by making dynamic processes perceptible that are difficult or impossible to interpret visually alone. Rather than treating sonification as an auxiliary feature, we develop principled computational frameworks that integrate auditory interaction as a fundamental modality in future computer-assisted surgical systems. By combining computational modeling, physics-based simulation, multisensory interaction, artificial intelligence, and human-centered design, we aim to establish auditory intelligence as a core component of the next generation of computer-assisted medicine.
Research Areas
Our work has evolved from exploratory musical mappings to physics-based, interactive, and sensing-driven sonification systems.
From Tissue to Sound
Physics-Based Sonification of Tool–Tissue Interaction
Our central research framework transforms simulated and measured tissue interactions into physically and perceptually meaningful sound. It establishes the computational foundation for our current work in interactive medicine. This project introduces a physics-driven framework that transforms biomechanical tissue properties into meaningful auditory feedback during robotic interventions. By integrating medical imaging, computational modeling, and interactive sonification, it enables surgeons to interpret tissue behavior through sound, supporting multimodal perception and decision-making in complex surgical environments.
Surgical Soundtracks
Our early exploration of how music can serve as a medium for medical sonification in surgery
Surgical Soundtracks represents our early exploration of musical sonification in image-guided interventions. The project investigated how surgical motion and procedural data could be transformed into expressive musical representations while preserving clinically meaningful information. By mapping instrument trajectories and surgical events to musical structures, it demonstrated the potential of music as an intuitive medium for data interpretation, education, and public engagement, laying the conceptual foundation for our later research on interactive, physics-driven surgical sonification.
Spatial Sonification
Auditory Guidance for Surgical Navigation
Spatial Sonification explores how three-dimensional auditory cues can enhance perception and navigation in complex medical environments. By leveraging spatial audio and immersive sound rendering, the project encodes the position, orientation, and movement of surgical instruments and anatomical structures into intuitive auditory feedback. This work investigates how spatial hearing can complement visual information, supporting more natural interaction and improved situational awareness during image-guided and robot-assisted procedures.
- Roodaki, Hessam, Navid Navab, Abouzar Eslami, Christopher Stapleton, and Nassir Navab. "Sonifeye: Sonification of visual information using physical modeling sound synthesis." IEEE transactions on Visualization and Computer Graphics 23, no. 11 (2017): 2366-2371.
- Matinfar, Sasan; Salehi, Mehrdad; Suter, Daniel; Seibold, Matthias; Dehghani, Shervin; Navab, Navid; Wanivenhaus, Florian; Fürnstahl, Philipp; Farshad, Mazda; Navab, Nassir: Sonification as a reliable alternative to conventional visual surgical navigation. Scientific Reports 13 (1), 2023, 5930
C. Current Research
BioSonix
Interactive Tool–Tissue Sonification
Physics-based auditory feedback for perceiving tissue deformation and tool–tissue interaction in real time.
iOCT Sonification
Auditory Interaction Awareness in Microsurgery
Real-time sonification of intraoperative OCT information for ophthalmic microsurgery and subretinal intervention.
Vibroacoustic Sensing
Sensing Physical Interaction Through Sound
Investigating vibroacoustic signals as a source of information for tissue characterization, interaction sensing, and future auditory feedback systems.
Team
Prof. Nassir Navab
Supervisor
Dr. Sasan Matinfar
Research Lead
Dr. Veronica Ruozzi
Senior Researcher
Dr. Wenhan Sun
Senior Researcher
Hannes Firzlaff
Doctoral Researcher
Research Partners
We collaborate with leading academic institutions, hospitals, and industry partners to advance auditory interaction technologies for computer-assisted medicine.
Academic
- Technical University of Dresden
- LMU University Hospital – Department of Ophthalmology
- Politecnico di Milano
- Balgrist University Hospital
- Otto von Guericke University Magdeburg
- Bielefeld University
- Suzhou University of Science and Technology
- The Chinese University of Hong Kong
- Concordia University
Industrial
- SURAG Medical
- SynthesEyes
Open Positions
We are always looking for motivated students, PhD candidates and collaborators.
Contact Person
Dr. Sasan Matinfar, sasan.matinfar@tum.de