Open Topics

We offer multiple Bachelor/Master theses, Guided Research projects and IDPs in the area of data mining/machine learning. A non-exhaustive list of open topics is listed below.

If you are interested in a thesis or a guided research project, please send your CV and transcript of records to Prof. Stephan Günnemann via email and we will arrange a meeting to talk about the potential topics.

Graph Neural Networks for Spatial Transcriptomics

Type: Master's Thesis

Prerequisites:

• Strong machine learning knowledge
• Proficiency with Python and deep learning frameworks (PyTorch, TensorFlow, JAX)
• Knowledge of graph neural networks (e.g., GCN, MPNN)
• Optional: Knowledge of bioinformatics and genomics

Description:

Spatial transcriptomics is a cutting-edge field at the intersection of genomics and spatial analysis, aiming to understand gene expression patterns within the context of tissue architecture. Our project focuses on leveraging graph neural networks (GNNs) to unlock the full potential of spatial transcriptomic data. Unlike traditional methods, GNNs can effectively capture the intricate spatial relationships between cells, enabling more accurate modeling and interpretation of gene expression dynamics across tissues. We seek motivated students to explore novel GNN architectures tailored for spatial transcriptomics, with a particular emphasis on addressing challenges such as spatial heterogeneity, cell-cell interactions, and spatially varying gene expression patterns.

Contact: Filippo Guerranti, Alessandro Palma

References:

• Cell clustering for spatial transcriptomics data with graph neural network
• Unsupervised spatially embedded deep representation of spatial transcriptomics
• SpaGCN: Integrating gene expression, spatial location and histology to identify spatial domains and spatially variable genes by graph convolutional network
• DeepST: identifying spatial domains in spatial transcriptomics by deep learning
• Deciphering spatial domains from spatially resolved transcriptomics with an adaptive graph attention auto-encoder

• GCNG: graph convolutional networks for inferring gene interaction from spatial transcriptomics data

Generative Models for Drug Discovery

Type: Mater Thesis / Guided Research

Prerequisites:

• Strong machine learning knowledge
• Proficiency with Python and deep learning frameworks (PyTorch or TensorFlow)
• Knowledge of graph neural networks (e.g. GCN, MPNN)
• No formal education in chemistry, physics or biology needed!

Description:

Effectively designing molecular geometries is essential to advancing pharmaceutical innovations, a domain which has experienced great attention through the success of generative models. These models promise a more efficient exploration of the vast chemical space and generation of novel compounds with specific properties by leveraging their learned representations, potentially leading to the discovery of molecules with unique properties that would otherwise go undiscovered. Our topics lie at the intersection of generative models like diffusion/flow matching models and graph representation learning, e.g., graph neural networks. The focus of our projects can be model development with an emphasis on downstream tasks (e.g., diffusion guidance at inference time) and a better understanding of the limitations of existing models.

Contact: Johanna Sommer, Leon Hetzel

References:

Equivariant Diffusion for Molecule Generation in 3D

Equivariant Flow Matching with Hybrid Probability Transport for 3D Molecule Generation

Structure-based Drug Design with Equivariant Diffusion Models

Efficient Machine Learning: Pruning, Quantization, Distillation, and More - DAML x Pruna AI

Type: Master's Thesis / Guided Research / Hiwi

Prerequisites:

• Strong knowledge in machine learning
• Proficiency with Python and deep learning frameworks (TensorFlow or PyTorch)

Description:

The efficiency of machine learning algorithms is commonly evaluated by looking at target performance, speed and memory footprint metrics. Reduce the costs associated to these metrics is of primary importance for real-world applications with limited ressources (e.g. embedded systems, real-time predictions). In this project, you will work in collaboration with the DAML research group and the Pruna AI startup on investigating solutions to improve the efficiency of machine leanring models by looking at multiple techniques like pruning, quantization, distillation, and more.

Contact: Bertrand Charpentier

References:

1. The Efficiency Misnomer
2. A Gradient Flow Framework for Analyzing Network Pruning
3. Distilling the Knowledge in a Neural Network
4. A Survey of Quantization Methods for Efficient Neural Network Inference

Deep Generative Models

Type: Master Thesis / Guided Research

Prerequisites:

• Strong machine learning and probability theory knowledge
• Proficiency with Python and deep learning frameworks (TensorFlow or PyTorch)
• Knowledge of generative models and their basics (e.g., Normalizing Flows, Diffusion Models, VAE)
• Optional: Neural ODEs/SDEs, Optimal Transport, Measure Theory

Description:

With recent advances, such as Diffusion Models, Transformers, Normalizing Flows, Flow Matching, etc., the field of generative models has gained significant attention in the machine learning and artificial intelligence research community. However, many problems and questions remain open, and the application to complex data domains such as graphs, time series, point processes, and sets is often non-trivial. We are interested in supervising motivated students to explore and extend the capabilities of state-of-the-art generative models for various data domains.

Contact: Marcel Kollovieh, David Lüdke

References:

• Flow Matching for Generative Modeling
• Auto-Encoding Variational Bayes
• Denoising Diffusion Probabilistic Models 
• Structured Denoising Diffusion Models in Discrete State-Spaces

Active Learning for Multi Agent 3D Object Detection 

Type: Master's Thesis 
Industrial partner: BMW 

Prerequisites: 

• Strong knowledge in machine learning 
• Knowledge in Object Detection 
• Excellent programming skills 
• Proficiency with Python and deep learning frameworks (TensorFlow or PyTorch) 

Description: 

In autonomous driving, state-of-the-art deep neural networks are used for perception tasks like for example 3D object detection. To provide promising results, these networks often require a lot of complex annotation data for training. These annotations are often costly and redundant. Active learning is used to select the most informative samples for annotation and cover a dataset with as less annotated data as possible.   

The objective is to explore active learning approaches for 3D object detection using combined uncertainty and diversity based methods.  

Contact: Sebastian Schmidt

References: 

1. Exploring Diversity-based Active Learning for 3D Object Detection in Autonomous Driving  
2. Efficient Uncertainty Estimation for Semantic Segmentation in Videos  
3. KECOR: Kernel Coding Rate Maximization for Active 3D Object Detection
4. Towards Open World Active Learning for 3D Object Detection  

Graph Neural Networks

Type: Master's thesis / Bachelor's thesis / guided research

Prerequisites:

• Strong machine learning knowledge
• Proficiency with Python and deep learning frameworks (TensorFlow or PyTorch)
• Knowledge of graph neural networks (e.g. GCN, MPNN)
• Knowledge of graph/network theory

Description:

Graph neural networks (GNNs) have recently achieved great successes in a wide variety of applications, such as chemistry, reinforcement learning, knowledge graphs, traffic networks, or computer vision. These models leverage graph data by updating node representations based on messages passed between nodes connected by edges, or by transforming node representation using spectral graph properties. These approaches are very effective, but many theoretical aspects of these models remain unclear and there are many possible extensions to improve GNNs and go beyond the nodes' direct neighbors and simple message aggregation.

Contact: Simon Geisler

References:

1. Semi-supervised classification with graph convolutional networks
2. Relational inductive biases, deep learning, and graph networks
3. Diffusion Improves Graph Learning
4. Weisfeiler and leman go neural: Higher-order graph neural networks
5. Reliable Graph Neural Networks via Robust Aggregation

Physics-aware Graph Neural Networks

Type: Master's thesis / guided research

Prerequisites:

• Strong machine learning knowledge
• Proficiency with Python and deep learning frameworks (JAX or PyTorch)
• Knowledge of graph neural networks (e.g. GCN, MPNN, SchNet)
• Optional: Knowledge of machine learning on molecules and quantum chemistry

Description:

Deep learning models, especially graph neural networks (GNNs), have recently achieved great successes in predicting quantum mechanical properties of molecules. There is a vast amount of applications for these models, such as finding the best method of chemical synthesis or selecting candidates for drugs, construction materials, batteries, or solar cells. However, GNNs have only been proposed in recent years and there remain many open questions about how to best represent and leverage quantum mechanical properties and methods.

Contact: Nicholas Gao

References:

1. Directional Message Passing for Molecular Graphs
2. Neural message passing for quantum chemistry
3. Learning to Simulate Complex Physics with Graph Network
4. Ab initio solution of the many-electron Schrödinger equation with deep neural networks
5. Ab-Initio Potential Energy Surfaces by Pairing GNNs with Neural Wave Functions
6. Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds

Robustness Verification for Deep Classifiers

Type: Master's thesis / Guided research

Prerequisites:

• Strong machine learning knowledge (at least equivalent to IN2064 plus an advanced course on deep learning)
• Strong background in mathematical optimization (preferably combined with Machine Learning setting)
• Proficiency with python and deep learning frameworks (Pytorch or Tensorflow)
• (Preferred) Knowledge of training techniques to obtain classifiers that are robust against small perturbations in data

Description: Recent work shows that deep classifiers suffer under presence of adversarial examples: misclassified points that are very close to the training samples or even visually indistinguishable from them. This undesired behaviour constraints possibilities of deployment in safety critical scenarios for promising classification methods based on neural nets. Therefore, new training methods should be proposed that promote (or preferably ensure) robust behaviour of the classifier around training samples.

Contact: Aleksei Kuvshinov

References (Background):

1. Intriguing properties of neural networks
2. Explaining and harnessing adversarial examples
3. SoK: Certified Robustness for Deep Neural Networks

References:

1. Certified Adversarial Robustness via Randomized Smoothing
2. Formal guarantees on the robustness of a classifier against adversarial manipulation
3. Towards deep learning models resistant to adversarial attacks
4. Provable defenses against adversarial examples via the convex outer adversarial polytope
5. Certified defenses against adversarial examples
6. Lipschitz-margin training: Scalable certification of perturbation invariance for deep neural networks

Uncertainty Estimation in Deep Learning

Type: Master's Thesis / Guided Research

Prerequisites:

• Strong knowledge in machine learning
• Strong knowledge in probability theory
• Proficiency with Python and deep learning frameworks (TensorFlow or PyTorch)

Description:

Safe prediction is a key feature in many intelligent systems. Classically, Machine Learning models compute output predictions regardless of the underlying uncertainty of the encountered situations. In contrast, aleatoric and epistemic uncertainty bring knowledge about undecidable and uncommon situations. The uncertainty view can be a substantial help to detect and explain unsafe predictions, and therefore make ML systems more robust. The goal of this project is to improve the uncertainty estimation in ML models in various types of task.

Contact: Tom Wollschläger, Dominik Fuchsgruber, Bertrand Charpentier

References:

1. Can You Trust Your Model’s Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
2. Predictive Uncertainty Estimation via Prior Networks
3. Posterior Network: Uncertainty Estimation without OOD samples via Density-based Pseudo-Counts
4. Evidential Deep Learning to Quantify Classification Uncertainty
5. Weight Uncertainty in Neural Networks

Hierarchies in Deep Learning

Type: Master's Thesis / Guided Research

Prerequisites:

• Strong machine learning knowledge
• Proficiency with Python and deep learning frameworks (TensorFlow or PyTorch)

Description:

Multi-scale structures are ubiquitous in real life datasets. As an example, phylogenetic nomenclature naturally reveals a hierarchical classification of species based on their historical evolutions. Learning multi-scale structures can help to exhibit natural and meaningful organizations in the data and also to obtain compact data representation. The goal of this project is to leverage multi-scale structures to improve speed, performances and understanding of Deep Learning models.

Contact: Marcel Kollovieh, Bertrand Charpentier

References:

1. Tree Sampling Divergence: An Information-Theoretic Metricfor Hierarchical Graph Clustering
2. Hierarchical Graph Representation Learning with Differentiable Pooling
3. Gradient-based Hierarchical Clustering
4. Gradient-based Hierarchical Clustering using Continuous Representations of Trees in Hyperbolic Space