Zum Inhalt springen
Technische Universität München

Vision-Language Group

About

The Vision-Language Group is led by Dr. Matthias Keicher at the Chair for Computer-Aided Medical Procedures & Augmented Reality (CAMP) of Prof. Dr. Nassir Navab, Technical University of Munich (TUM). We work on multimodal language models for clinical reasoning: how a model turns medical images, reports and patient records into a diagnostic assessment, and how it states what it does not know.

Our current focus is calibrated uncertainty as a training signal (Rewarding Doubt, ConRad) and reinforcement-learned clinical agents that decide which evidence to request next (LA-CDM), evaluated with clinical partners at TUM University Hospital and external clinical partners. Earlier work established language-grounded structured reporting (FlexR, Xplainer, Rad-ReStruct), radiology vision-language assistants (RaDialog) and patient-pathway modelling from electronic health records (EHR2Path).

Contact Person / Group Coordination

Matthias Keicher - matthias.keicher@tum.de

Group Members

Current Research Projects

Recent Publications

MICCAI 2026Calibrated Confidence Expression for Radiology Report Generation
TMLR 2026From EHRs to Patient Pathways: Scalable Modeling of Longitudinal Health Trajectories with LLMs
ICLR 2026Language Agents for Hypothesis-driven Clinical Decision Making with Reinforcement Learning
ICLR 2026Rewarding Doubt: A Reinforcement Learning Approach to Calibrated Confidence Expression of Large Language Models
npj Digital MedicineSpecialized foundation models for intelligent operating rooms
CVPR 2025MM-OR: A Large Multimodal Operating Room Dataset for Semantic Understanding of High-Intensity Surgical Environments
MIDL 2025RaDialog: A Large Vision-Language Model for Radiology Report Generation and Conversational Assistance
MedAGI 2024MAGDA: Multi-agent Guideline-Driven Diagnostic Assistance
MICCAI 2024ORacle: Large Vision-Language Models for Knowledge-Guided Holistic OR Domain Modeling
MICCAI 2023Rad-ReStruct: A Novel VQA Benchmark and Method for Structured Radiology Reporting
MICCAI 2023Xplainer: From X-Ray Observations to Explainable Zero-Shot Diagnosis
GRAIL 2023Prior-RadGraphFormer: A Prior-Knowledge-Enhanced Transformer for Generating Radiology Graphs from X-Rays
MIDL 2023FlexR: Few-shot Classification with Language Embeddings for Structured Reporting of Chest X-rays

 

Research Partners

Clinical partners
TUM University Hospital Klinikum rechts der Isar (Radiology, Neuroradiology, Thoracic Surgery, Toxicology, Dermatology)
University Hospital Augsburg (Radiology)
University Medical Center Hamburg-Eppendorf (UKE), Radiology
LMU University Hospital

Academic partners
Munich Center for Machine Learning (MCML)
Stanford University, School of Medicine
Harvard Medical School
FAU Erlangen-Nürnberg
Sharif University of Technology

Industry partners
NVIDIA
ImFusion GmbH
Jacobian (formerly Smart Reporting GmbH)
deepc GmbH

Links

Location

IFL: Interdisziplinäres Forschungslabor

Klinikum rechts der Isar
Ismaninger Str. 22
81675 München
Room: 01.01.3a-c (basement floor)
Phone: +49 (89) 4140-6457
Fax: +49 (89) 4140-6458

To top