Multimodal Health Chatbot Interface for Doctors and Patients in Dementia Care

Abstract:

Digital assistants for healthcare are capable of collecting large volumes of complex personal health data on a daily basis, including metrics such as blood pressure, step count, and cognitive test scores. However, analyzing and extracting meaningful insights from these data points remains a difficult and time-consuming process for doctors and caregivers.
 

This thesis explores the development of a conversational AI system that enables both medical professionals and patients to interact with their personal health data through natural language. Built within the context of the AssistD project and integrated with the existing health assistant platform from ALMA PHIL, the chatbot will retrieve relevant information from structured databases and generate concise, personalized responses, complemented by dynamically generated visualizations.
 

By leveraging natural language processing and multimodal output, this system aims to simplify the interpretation of longitudinal health data, support diagnostic and therapeutic decision-making, and enhance communication among stakeholders. A special focus will be placed on an extensive evaluation with real human users, ensuring response accuracy, accessibility, and transparency. The thesis will build upon an existing solution, and German language skills are helpful for interacting with participants.
 

Research Questions:

RQ1: How can a conversational interface be robustly designed to retrieve, summarize, and visualize personal health data in a way that meets the diverse needs of patients, caregivers, and healthcare professionals?

RQ2: What are the key challenges and strategies for optimizing the performance of the underlying models to ensure accurate, context-aware, and real-time responses to natural language queries?

RQ3: How do users perceive the interpretability, usability, and trustworthiness of the chatbot’s outputs, and how can user feedback inform iterative improvements, particularly in the context of dementia care?