Open Thesis Topic: Informed Machine Learning meets Mechanistic Interpretability
Background
Informed Machine Learning integrates prior knowledge (such as physical laws, logic rules, knowledge graphs, simulation results) explicitly into the ML pipeline, rather than relying on data alone. This can happen at various stages: in the training data, the model architecture, the learning algorithm, or the final model. The goal is better generalization and consistency with domain knowledge, especially when data is scarce.
Mechanistic Interpretability goes the other way: it tries to reverse-engineer an already-trained model's internal computations into human-understandable algorithms and concepts, often by identifying "circuits" - subnetworks responsible for specific behaviors.
These two fields are natural counterparts: one puts knowledge into a model, the other extracts knowledge back out. Combining them opens questions like whether injected knowledge is actually reflected in a model's internal computations, and whether interpretability findings can themselves become a new source of prior knowledge. This intersection is still largely unexplored.
Objectives
Possible directions include:
- Checking whether informed ML models truly internalize injected knowledge, using interpretability tools
- Comparing internal circuits of models trained with vs. without domain knowledge
- Using discovered circuits/features as a new knowledge source to guide training
- Detecting mismatches between intended prior knowledge and what the model actually computes
- Domain case studies (e.g., physics, healthcare) combining both perspectives
The exact topic will be scoped together based on your interests.
Requirements
- Solid ML/deep learning fundamentals
- Interest in interpretability and/or knowledge-guided modeling
- Python programming (PyTorch or similar)
- Comfort reading recent, fast-moving literature (often preprints)
Introductory Literature
- Von Rueden et al. (2021). Informed Machine Learning—A Taxonomy and Survey of Integrating Prior Knowledge into Learning Systems. IEEE Transactions on Knowledge and Data Engineering, 1–1. https://doi.org/10.1109/TKDE.2021.3079836
- Bereska, L., & Gavves, E. (2024). Mechanistic Interpretability for AI Safety—A Review (arXiv:2404.14082). arXiv. https://doi.org/10.48550/arXiv.2404.14082
- Olah, C., Cammarata, N., Schubert, L., Goh, G., Petrov, M., & Carter, S. (2020). Zoom In: An Introduction to Circuits. Distill, 5(3), e00024.001. https://doi.org/10.23915/distill.00024.001
- A Mathematical Framework for Transformer Circuits. (n.d.). Retrieved July 7, 2026, from https://transformer-circuits.pub/2021/framework/index.html