Image Synthesis - Bildsynthese (IN2015)

Prof. Dr. Rüdiger Westermann

Contents

This lecture covers the fundamentals of photorealistic and interactive image synthesis. We examine modern GPU design, rasterization and ray-tracing APIs, and new rendering pipeline stages such as task and mesh shaders. Selected graphics algorithms and their efficient implementation on state-of-the-art GPUs are discussed. We then focus on 3D shape models beyond classical triangle meshes, such as voxel grids and scattered point clouds, including their specific rendering techniques. Moving beyond real-time graphics, we introduce the governing equations of physics-based light transport and demonstrate effective solution methods - such as radiosity, photon mapping, and path tracing - for simulating global illumination. Finally, we cover modern differentiable and neural rendering approaches, including NeRFs and 3D Gaussian Splatting, followed by an introduction to neural networks and deep learning methods for rendering, such as upscaling, denoising, and scene representation networks, as well as modern APIs like Nvidia DLSS 5 and AMD FSR 4.

This lecture will be held in English. 

Topics

• Interactive rendering (i.e. rasterization and local illumination)
  • Graphics APIs and shaders
  • GPUs
  • Normal mapping, deferred shading, ambient occlusion
  • Order independent transparency and depth-peeling
• Voxel grids and scattered point clouds
  • Fast grid traversal
  • High-quality point rendering
• Physics based rendering (i.e. global illumination)
  • Radiometric quantities, radiance equation
  • Radiosity
  • Photon Mapping
  • Path-tracing
  • Volume rendering
• Differentiable and neural rendering
  • Forward vs. inverse rendering
  • NeRFs and 3DGS
  • Introduction to neural networks
  • Deep learning for upscaling, denoising, and radiance inference

Literature

• Pharr, Humphreys: Physically Based Rendering, Second Edition
• Cohen, Wallace: Radiosity and Realistic Image Synthesis
• Möller, Haines: Real-Time Rendering, 2nd edition, A.K. Peters LTD
• Eberly: 3D Game Engine Design, Morgan Kaufmann
• DeLoura et. al: Game Programming Gems 1 - 3, Charles River Media
• Foley, Van Dam, Feiner, Hughes: Computer Graphics: Principles and Practice, Addison-Wesley, 2nd edition in C
• Watt, Watt: Computer Graphics, Addison-Wesley
• Glassner: Principles of digital image synthesis, Morgan Kaufman
• Encarnaçao, Klein, Strasser: Graphische Datenverarbeitung, 4. Auflage, Oldenburg Verlag
• Griebel, Bungartz, Zenger: Computer Graphik, Vieweg Verlag

Assignments

There are no separate exercise classes for this lecture.