Master-Seminar - Deep Learning in Physics (IN2107, IN0014)

Prof. Dr. Nils Thuerey , Nilam Tathawadekar , Shuvayan Brahmachary

Master-Seminar - Deep Learning in Physics (IN2107, IN0014)

Time, Place	Wednesdays 14:00-16:00 in room: MI 02.13.010
Begin	19th., October 2022
Prerequisites	Introduction to Deep Learning

Information

https://ge.in.tum.de/teaching/seminars-ws-20-21

Content

Using deep learning methods for physical problems is a very quickly developing area of research. The research group of Prof. Thuerey has studied learning-based methods for Navier-Stokes problems and fluid flow applications in recent years, examples of which include learning latent-spaces for physical predictions, generative adversarial networks with temporal coherence, and the inference of Reynolds-averaged Navier-Stokes flows around airfoils. Beyond these physics-based deep learning works of the Thuerey group, this seminar will give an overview of recent developments in the field.

In this course, students will autonomously investigate recent research about machine learning techniques in the physical simulation area. Independent investigation for further reading, critical analysis, and evaluation of the topic are required.

Requirements

Participants are required to first read the assigned paper and start writing a report. This will help you prepare for your presentation.

Attendance

It is only allowed to miss a single time-slot. Missing a second one means failing the seminar. If you have to miss any, please let us know in advance.

Advisor

An advisor is assigned to each one with the paper.
Two weeks before the talk there will be a mandatory meeting with your advisor to review the report and discuss the structure of the presentation.

Report

A short report (4 pages max., excluding references in the ACM SIGGRAPH TOG format (acmtog) - you can download the precompiled latex template) should be prepared before the meeting with the advisor.
Guideline: You can begin with writing a summary of the work you present as a start point; but, it would be better if you focus more on your own research rather than just finishing with the summary of the paper. We, including you, are not interested in revisiting the work done before; it is more meaningful if you make an effort to put your own reasoning about the work, such as pros and cons, limitation, possible future work, your own ideas for the issues, etc.

Presentation (slides)

The participants have to present their topics in a talk (in English), which should last 30 minutes. Don't put too many technical details into the talk, make sure the audience gets the paper's main idea. Be prepared to answer questions regarding the technical details, you could prepare backup slides for that.
Afterwards, a short discussion session will follow.
Plagiarism is important; please do not simply copy the original authors' slides. You can certainly refer to them.
The semi-final slides (PDF) should be sent one week before the talk; otherwise, the talk will be canceled. We strongly encourage you to finalize the semi-final version as far as possible. We will take a look at the version and give feedback. You can revise your slides until your presentation.
Be ready in advance. We suggest testing the machines you are going to use before the lecture starts. You can bring your laptop or ask us one (also any converter you need for the projector) in advance. A laser pointer will be provided, so you can use if you want.
The final slides and report should be sent after the talk.

Preliminary Schedule

Paper list sent via Email

Send preferred 5 topics/papers by 05 September 2021 (Sunday 23:59)

- 20.10 Kickoff (Introduction lecture)

- 27.10 No meeting

- 3.11 No meeting

- 10.11 Meeting #1 (First presentation)

- 17.11 Meeting #2

- 24.11 Meeting #3

- 1.12 Meeting #4

- 8.12 Meeting #5

- 15.12 Meeting #6

- 22.12 No meeting

- 12.1 Meeting #7

- 19.1 Meeting #8

- 26.1 Meeting #9 (if necessary!)

Paper list

No.	Paper	Date	First name	Last name	Supervisor
1	Combining Differentiable PDE Solvers and Graph Neural Networks for Fluid Flow Prediction	10 November	Louis	Dorge de Vacher de Saint Géran	Chen
2	Machine learning accelerated computational fluid dynamics	10 November	Tianhao	Lin	Holzschuh
3	Assessment of unsteady flow predictions using hybrid deep learning based reduced-order models
4	Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers	17 November	Annalena	Kofler	Chen
5	Deep learning methods for super-resolution reconstruction of turbulent flows
6	Transfer learning for nonlinear dynamics and its application to fluid turbulence https://arxiv.org/pdf/2009.01407.pdf	17 November	Ege	Ahmed	Holzschuh
7	Learning to control PDEs with differentiable physics https://arxiv.org/pdf/2001.07457.pdf	24 November	Anis	Yaich	Chen
8	Discovering physical concepts with neural networks https://arxiv.org/pdf/1807.10300.pdf	24 November	Ajla	Karisik	Holzschuh
9	Neural Ordinary Differential Equations https://arxiv.org/pdf/1806.07366.pdf	1 December	Robert	Hajda	Chen
10	Hamiltonian Neural Networks	1 December	José Miguel	Ferreira Henriques	Holzschuh
11	Physics Informed Deep Learning: Data-driven Solutions of Nonlinear Partial Differential Equations https://arxiv.org/pdf/1711.10561.pdf	8 December	Christina	Nuss-Brill	Holzschuh
12	Model identification of reduced order fluid dynamics systems using deep learning
13	Deep learning for universal linear embeddings of nonlinear dynamics https://www.nature.com/articles/s41467-018-07210-0.pdf	8 December	Qing	Sun	Chen
14	tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow https://arxiv.org/pdf/1801.09710	15 December	Mert	Ülker	Chen
15	Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations	15 December	Stefan	Frisch	Holzschuh
16	Lagrangian Fluid Simulation with Continuous Convolutions https://openreview.net/pdf?id=B1lDoJSYDH
17	Solving high-dimensional partial differential equations using deep learning https://www.pnas.org/content/115/34/8505
18	Learning data-driven discretizations for partial differential equations https://www.pnas.org/content/pnas/116/31/15344.full.pdf
19	SPNets: Differentiable Fluid Dynamics for Deep Neural Networks https://arxiv.org/pdf/1806.06094.pdf	12 Jan. 2022	Anirudh Narayanan	Balaraman	Chen
20	Augmenting Physical Models with Deep Networks for Complex Dynamics Forecasting https://arxiv.org/abs/2010.04456	12 Jan. 2022	Tobias	Neumeier	Holzschuh
21	Discovering Symbolic Models from Deep Learning with Inductive Biases https://arxiv.org/pdf/2006.11287.pdf	19 Jan. 2022	Andreas	Stöckeler	Chen
22	Differentiable Strong Lensing: Uniting Gravity and Neural Nets through Differentiable Probabilistic Programming https://arxiv.org/pdf/1910.06157.pdf	19 Jan. 2022	Nadiia	Matsko	Holzschuh
23	DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators arxiv.org/abs/1910.03193

Papers

Date	Presenter	Paper
Nov. 20	Sagar Garg	2016, Tompson et al., Accelerating Eulerian Fluid Simulation With Convolutional Networks, arXiv.org
Dec. 11	Konrad Eder	2018, Schenck, SPNets: Differentiable Fluid Dynamics for Deep Neural Networks, arXiv.org
-	-	2015, Ladicky et al., Data-driven Fluid Simulations using Regression Forests, ACM Trans. Graph.
Nov. 27	Stephen Ryan	2018, Kim et al., Deep Fluids: A Generative Network for Parameterized Fluid Simulations, arXiv.org
-	-	2016, Holden et al., A Deep Learning Framework for Character Motion Synthesis and Editing, ACM Trans. Graph.
Nov. 6	Achraf Aroua	2018, Pathak et al., Model-Free Prediction of Large Spatiotemporally Chaotic Systems from Data: A Reservoir Computing Approach, Physical review letters
Dec. 18	Keerthi Gaddameedi	2018, Y. Xie et al., tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow, ACM Trans. Graph.
Nov. 20	Mohamed Attia	2016, C. Yang et al., Data-driven projection method in fluid simulation, Computer Animation and Virtual Worlds
Nov. 27	Jan Luca Watter	2018, Bailey et al., Fast and deep deformation approximations, ACM Trans. Graph.
Dec. 11	Amir Nourinia	2019, Li et al., Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids, ICLR 2019 Conference
Nov. 13	Max Oberberger	2019, Thuerey et al., Deep Learning Methods for Reynolds-Averaged Navier-Stokes Simulations of Airfoil Flows, arXiv.org
Dec. 18	Shucheng Yang	2017, Um et al., Liquid Splash Modeling with Neural Networks, Computer Graphics Forum
Nov. 6	Anna Maria Geissinger	2019, Raissi et al. Physics-informed Neural Networks: A Deep Learning Framework for Solving Forward and Inverse Problems Involving Nonlinear Partial Differential Equations, Journal of Computational Physics
-	-	2019, Zhu et al., Machine learning Methods for Turbulence Modeling in Subsonic Flows Around Airfoils, AIP
Nov. 13	Roland Konlechner	2018, Zhang et al., Application of Convolutional Neural Network to Predict Airfoil Lift Coefficient, arXiv.org
-	-	2019, Raissi et al., Deep learning of vortex-induced vibrations, Journal of Fluid Mechanics
Dec. 4	David Elias Drothier	2019, Hu et al., ChainQueen: A Real-Time Differentiable Physical Simulator for Soft Robotics, arXiv.org
Dec. 4	Anjie Guo	2019, Xu et al., DensePhysNet: Learning Dense Physical Object Representations via Multi-step Dynamic Interaction, RSS 2019

You can access the papers through TUM library's eAccess.

Papers

Discovering Nonlinear PDEs from Scarce Data with Physics-encoded Learning

Model identification of reduced order fluid dynamics systems using deep learning

Direct shape optimization through deep reinforcement learning

Numerical investigation of minimum drag profiles in laminar flow using deep learning surrogates

Data-driven nonlinear aeroelastic models of morphing wings for control

Accelerating Eulerian Fluid Simulation With Convolutional Networks

Deep Learning Methods for Reynolds-Averaged Navier-Stokes Simulations of Airfoil Flows

Deep learning methods for super-resolution reconstruction of turbulent flows

Physics Informed Deep Learning: Data-driven Solutions of Nonlinear Partial Differential Equations

Learning data-driven discretizations for partial differential equations

Neural Ordinary Differential Equations

Learning to control PDEs with differentiable physics

Combining Differentiable PDE Solvers and Graph Neural Networks for Fluid Flow Prediction

CFDNet: a deep learning-based accelerator for fluid simulations

Machine learning accelerated computational fluid dynamics

Learned Coarse Models for Efficient Turbulence Simulation

Lagrangian Fluid Simulation with Continuous Convolutions

Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers

Solving high-dimensional partial differential equations using deep learning

Transfer learning for nonlinear dynamics and its application to fluid turbulence

Towards Physics-informed Deep Learning for Turbulent Flow Prediction

Message Passing Neural PDE Solvers

Neural Solvers for Fast and Accurate Numerical Optimal Control

Presentations

Presentation date	Presenter	Paper	Advisor
09.11.2022	K. Bali	Accelerating Eulerian Fluid Simulation With Convolutional Networks	Nilam
09.11.2022	K. Selim	Data-driven nonlinear aeroelastic models of morphing wings for control	Nilam
16.11.2022	A. Pimpalkar	Lagrangian Fluid Simulation with Continuous Convolutions	Shuvayan
16.11.2022	Y. Shehata	Direct shape optimization through deep reinforcement learning	Shuvayan
23.11.2022	J. Oldenstädt	Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations	Shuvayan
23.11.2022	G. Gruhlke	Learning data-driven discretizations for partial differential equations	Nilam
30.11.2022	S. Javad	Deep Learning Methods for Reynolds-Averaged Navier-Stokes Simulations	Shuvayan
30.11.2022	D. Grothe	Deep learning methods for super-resolution reconstruction of turbulent flows	Nilam
07.12.2022	X. Luo	Neural Ordinary Differential Equations	Nilam
07.12.2022	A. Chumak	Learning to control PDEs with differentiable physics	Shuvayan
14.12.2022	M. Saeidy Pour	Machine learning accelerated computational fluid dynamics	Shuvayan
14.12.2022	G. Jing	CFDNet: a deep learning-based accelerator for fluid simulations	Shuvayan
21.12.2022	J. Lassen	Towards physics-informed deep learning for turbulent flow prediction	Shuvayan
21.12.2022	G. Kripka	Combining Differentiable PDE Solvers and Graph Neural Networks for Fluid Flow Prediction	Nilam
11.01.2023	F. Brandis	Learned Coarse Models for Efficient Turbulence Simulation	Nilam
11.01.2023	S. Mohamed	Solving high-dimensional partial differential equations using deep learning	Nilam

References

Thuerey group: List of Publications (including Physics-based Deep Learning works)
Book: Bishop, Pattern Recognition and Machine Learning
Book: Hastie et al., The Elements of Statistical Learning
Online: Nielsen, Neural Networks and Deep Learning
Online: Ruder, An Overview of Gradient Descent Optimization Algorithms