Dianye Huang

Research Interests

Dianye Huang is a research scientist at the Chair for Computer-Aided Medical Procedures (CAMP), Technische Universität München (TUM), advised by Prof. Nassir Navab. His research focuses on an intelligent robotic ultrasound imaging system, aiming to integrate multiple sensing modalities to achieve human-like, autonomous ultrasound scanning. This multidisciplinary topic lies at the intersection of robotics, computer vision, medical imaging, and machine learning. Accordingly, his research interests span intelligent robotic control, medical image analysis, robot learning, VLM/VLA for general manipulation tasks, and other techniques that can be transferred across domains and hold strong potential for advancing truly intelligent, clinically reliable, and patient-accepted medical robotic systems. 

Research interests:

• Medical Robotics: Robotic Ultrasound, Spatial Temporal Ultrasound Image Analysis
• Robot Learning: Embodied Agents empowered by large language/vision-language/action models (LLM/LVLM/VLA)
• Intelligent Robotic Control: physical Human-Robot / Robot-Environment Interaction

Curriculum Vitae

• Jun. 2025 -  now: Research scientist at CAMP, Technical University of Munich, Germany
• Sept. 2021 - Jun. 2025:  Ph.D. in computer science at CAMP, Technical University of Munich, Germany
• Sept. 2017 - Jun. 2020:  M.Eng. in Control Science and Engineering at South China University of Technology, Guangzhou, China
• Sept. 2013 - Jun. 2017:  B.Eng. in Automation at South China University of Technology, Guangzhou, China

Links and Profiles

Google Scholar       LinkedIn       GitHub 

News

• 12.2025: 1 paper early accepted by IPCAI2026, Nagoya
• 12.2025: <HIT Webinar> Invited Dr. Qianhui Men (Lecturer at the University of Bristol) to deliver a talk and hosted the session
• 10.2025: 1 paper accepted by IEEE Transactions on Robotics (TRO)
• 10.2025: 1 paper accepted by IEEE/ASME Transactions on Mechatronics
• 09.2025: Will serve as an Associate Editor for ICRA2026
• 08.2025: Received the IES SYPA Travel Award for IROS2025
• 08.2025: <HIT Webinar> Invited Prof. Jipeng Yan (Harbin Institute of Technology) to deliver a talk and hosted the session
• 07.2025: <HIT Webinar> Invited Prof. Guochen Ning (Tsinghua University) to deliver a talk and hosted the session
• 06.2025: 2 papers accepted by IROS2025, Hangzhou
• 06.2025: 1 paper accepted by MICCAI2025
• 06.2025: Student organizer of the 3rd Robotic-Assisted Medical Imaging (RAMI) workshop at ICRA2025, May, Atlanta, USA
• 06.2025: <HIT Webinar> Invited Dr. Zhen Chen (Yale University) to deliver a talk and hosted the session.
• 05.2025: Received the “Best Innovation Award” at the MRC symposium 2025
• 04.2025: <HIT Webinar> Invited Dr. Xihan Ma (Worcester Polytechnic Institute) to deliver a talk and hosted the session
• 04.2025: <HIT Webinar> Invited Prof. Yutong Xie (MBZUAI) to deliver a talk and hosted the session
• 02.2025: 1 paper accepted by IEEE Transactions on Medical Imaging
• 01.2025: 1 paper accepted by ICRA2025, Atlanta
• 01.2025: <HIT Webinar>Invited Prof. Liang Zhao (The University of Edinburgh) to deliver a talk and hosted the session
• 01.2025: <HIT Webinar>Invited Dr. Yicheng Wu (Monash University) to deliver a talk and hosted the session
• 12.2024: 1 paper early accepted by IPCAI2025, Berlin
• 11.2024: Join the Healthcare, Intelligence, Technology Webinar (HIT Webinar) organization team
• 05.2024: Student organizer of the 2nd Robotic-Assisted Medical Imaging (RAMI) workshop at ICRA2024, May, Yokohama, Japan
• 04.2024: 1 paper accepted by IEEE Transactions on Industrial Informatics
• 01.2024: 1 paper accepted by Annual Review of Control, Robotics, and Autonomous Systems
• 01.2024: 1 paper accepted by ICRA2024, Yokohama
• 12.2023: 1 paper accepted by IEEE Transactions on Automation Science and Engineering (presented at IROS2024)
• 06.2023: 1 paper accepted by IROS2023, Detroit
• 05.2023: Student organizer of the 1st Robotic-Assisted Medical Imaging (RAMI) workshop at ICRA2023, May, London, UK
• 01.2023: 1 paper accepted by ICRA2023, London
• 02.2023: 1 paper accepted by Robotics and Autonomous Systems
• 09.2021: Join CAMP

Student Projects

Feel free to reach out if you are looking for IDP, Guided Research Project, or Master Thesis and have any interesting ideas or topics related to, but not limited to, robotic ultrasound scanning, dynamic ultrasound imaging processing, ultrasound-based human-machine interaction, or embodied agents with LLM/LVLM/VLA. 

Year	Type	Titile	Student	Status
SS 2026	Master Thesis	Autonomous Robotic Ultrasound Scanning System	Pei Liu	ongoing
WS 2025/26	Master Thesis	Anatomy-grounded Vision Language Action Model for Ultrasound	Zhehao Duan	ongoing
WS 2025/26	Master Thesis	Vision-Language Model for Measurement Regression in Ultrasound	Jiamei Guo	ongoing
SS 2025	Course project	Continuous Needle Detection via Frequency-Inspired Features	J. Guo & Z. Duan	Early Accept by IPCAI
WS 2024/25	Master Thesis	Robotic Ultrasound Scanning System with Action Justifications	Ziping Cong	submitted to a conference
WS 2023/24	Master Thesis	Enhancing Needle Tip Detection using Motion Magnification Algorithms	Chenyang Li	Pub. in IEEE TMI

Publications

- 2026

Felix Duelmer, Mario Reidl, Nikolina-Alexia Fasoula, Nikoletta Katsouli, Dianye Huang, Yuan Bi, Raphael Salz, Rhiannon Lees, Zhongliang Jiang, Christoph Knappich, Heiko Wendorff, Daniela Branzan, Michael Kallmayer, Hans-Henning Eckstein, Nassir Navab, Angelos Karlas, “Translational Research and Clinical Trends in Robot-Assisted Vascular Surgery: A Narrative Overview, ” Handbook of Robotic and Image-Guided Surgery, 2026

- 2025

Dianye Huang, Nassir, Navab, Zhongliang Jiang, “Improving Robustness to Out-of-Distribution States in Imitation Learning via Deep Koopman-Boosted Diffusion Policy, ” IEEE Transactions on Robotics, 2025

Dianye Huang, Chenyang Li, Angelos Karlas, Xiangyu Chu, KW Samuel Au, Nassir Navab, Zhongliang Jiang, “VibNet: Vibration-Boosted Needle Detection in Ultrasound Images,” IEEE Transactions on Medical Imaging, 2025

Yameng Zhang, Dianye Huang, Max Q-H Meng, Nassir Navab, Zhongliang Jiang, “Freehand 3D Ultrasound Imaging: Sim-in-the-Loop Probe Pose Optimization via Visual Servoing, ” IEEE/ASME Transactions on Mechatronics, 2025

Xuesong Li, Dianye Huang, Yameng Zhang, Nassir Navab, Zhongliang Jiang, “Semantic scene graph for ultrasound image explanation and scanning guidance,”  MICCAI 2025

Zhongyu Chen, Chenyang Li, Xuesong Li, Dianye Huang, Zhongliang Jiang, Stefanie Speidel, Xiangyu Chu, KW Au, “Vibration-Based Energy Metric for Restoring Needle Alignment in Autonomous Robotic Ultrasound, ” IROS 2025

Yifan Zhang, Dianye Huang, Nassir Navab, Zhongliang Jiang, “Tactile-Guided Robotic Ultrasound: Mapping Preplanned Scan Paths for Intercostal Imaging, ” IROS 2025

Dianye Huang, Nassir Navab, Zhongliang Jiang, "Improving Probe Localization for Freehand 3D Ultrasound using Lightweight Cameras," ICRA 2025

Feng Li, Yuan Bi, Dianye Huang, Zhongliang Jiang, Nassir Navab, “Robotic CBCT meets robotic ultrasound,” International Journal of Computer Assisted Radiology and Surgery, 2025

- 2024

Dianye Huang, Chenguang Yang, Mingchuan Zhou, Angelos Karlas, Nassir Navab, Zhongliang Jiang, "Robot-assisted deep venous thrombosis ultrasound examination using virtual fixture," IEEE Transactions on Automation Science and Engineering, 2024

Shangding Gu, Dianye Huang, Muning Wen, Guang Chen, Alois Knoll, "Safe Multiagent Learning With Soft Constrained Policy Optimization in Real Robot Control," IEEE Transactions on Industrial Informatics, 2024

Yuan Bi, Zhongliang Jiang, Felix Duelmer, Dianye Huang, Nassir Navab, "Machine learning in robotic ultrasound imaging: Challenges and perspectives," , Annual Review of Control, Robotics, and Autonomous Systems, 2024

Guangyao Zhai, Xiaoni Cai, Dianye Huang, Yan Di, Fabian Manhardt, Federico Tombari, Nassir Navab, Benjamin Busam, "Sg-bot: Object rearrangement via coarse-to-fine robotic imagination on scene graphs, " ICRA 2024

- 2023

Dianye Huang, Yuan Bi, Nassir Navab, and Zhongliang Jiang, "Motion magnification in robotic sonography: enabling pulsation-aware artery segmentation, ", IROS 2023

Guangyao Zhai, Dianye Huang, Shun-Cheng Wu, HyunJun Jung, Yan Di, Fabian Manhardt, Federico Tombari, Nassir Navab, Benjamin Busam,  "Monograspnet: 6-dof grasping with a single rgb image", ICRA 2023

Yingbai Hu, Mingyang Cui, Jianghua Duan, Wenjun Liu, Dianye Huang, Alois Knoll, Guang Chen, "Model predictive optimization for imitation learning from demonstrations," in Robotics and Autonomous Systems(2023): 104381.

- 2021 and before

Dianye Huang, Chenguang Yang, Yongping Pan, Long Cheng, "Composite Learning Enhanced Neural Control for Robot Manipulator With Output Error Constraints," in IEEE Transactions on Industrial Informatics, vol. 17, no. 1, pp. 209-218, Jan. 2021.

Chenguang Yang, Dianye Huang, Wei He, Long Cheng, "Neural Control of Robot Manipulators With Trajectory Tracking Constraints and Input Saturation," in IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 9, pp. 4231-4242, Sept. 2021.

Hong Zhan, Dianye Huang, Zhaopeng Chen, Min Wang, Chenguang Yang. “Adaptive dynamic programming-based controller with admittance adaptation for robot–environment interaction,” International Journal of Advanced Robotic Systems. 2020;17(3).

Dianye Huang, Chenguang Yang, Ning Wang, Andy Annamalai, Chun-Yi Su. Online Robot Reference Trajectory Adaptation for Haptic Identification of Unknown Force Field. Int. J. Control Autom. Syst. 16, 318–326 2018

Dianye Huang, Chenguang Yang, Yongping Pan, Shilu Dai, Zhaojie Ju, "Composite Learning for Trajectory Tracking Control of Robot Manipulators with Output Constraints," 2018 Eighth International Conference on Information Science and Technology (ICIST), Cordoba, Granada, and Seville, Spain, 2018, pp. 301-306