Machine Learning

Tentative list of topics

• Introduction
  • What is machine learning?
  • Typical tasks in ML
• k-Nearest neighbors
  • kNN for classification and regression
  • Distance functions
  • Curse of dimensionality
• Decision trees
  • Constructing & pruning decision trees
  • Basics of information theory
• Probabilistic inference
  • Parameter estimation
  • Maximum likelihood principle
  • Maximum a posteriori
  • Full Bayesian approach
• Linear regression
  • Linear basis function models
  • Overfitting
  • Bias-variance tradeoff
  • Model selection
  • Regularization
• Linear classification
  • Perceptron algorithm
  • Generative / discriminative models for classification
  • Linear discriminant analysis
  • Logistic regression
• Optimization
  • Gradient-based methods
  • Convex optimization
  • Stochastic gradient descent
• Support vector machines
  • Maximum margin classification
  • Soft-margin SVM
  • Constrained optimization
• Kernel methods
  • Kernel trick
  • Kernelized linear regression
• Deep learning
  • Feedforward neural networks
  • Backpropagation
  • Advanced architectures
  • Adaptive step-size selection
• Dimensionality reduction
  • Principal component analysis
  • Singular value decomposition
  • Probabilistic PCA
• Mixture models
  • Gaussian mixture models
  • K-means
  • Topic models
  • EM algorithm
• Variational inference
  • Posterior inference in latent variable models
  • Mean-field approximation
  • Evidence lower bound

Literature

• Pattern Recognition and Machine Learning. Christopher Bishop. Springer-Verlag New York. 2006.
• Machine Learning: A Probabilistic Perspective. Kevin Murphy. MIT Press. 2012

Prerequisites

• Good understanding of Linear Algebra, Analysis, Probability and Statistics.
• Programming experience (preferably in Python).

Schedule

• Lecture:
  • Monday 10:00 - 11:45, room MW 0001
• Lecture / Practical session:
  • Tuesday 12:15 - 13:45, room MW 0001
• Homework discussion:
  • Wednesday 16:00 - 18:00, room MI 00.02.001
• Q&A session:
  • Wednesday 12:00 - 14:00, room MI 02.11.018

Organizational details

• Language: English
• Intended audience:
  • Master students of the Informatics department (Including Data Engineering & Analytics program).
  • Not available for Information Systems (Wirtschaftsinformatik) students.
  • Only TUM students are allowed to write the exam.
• New regulation: Please note that you can only include one of IN2064 / IN2332 in your curriculum
• 8 ECTS
• Grade bonus of 0.3 will be awarded to students who show sufficient work for at least 75% of the homework sheets. Note, that the grades 1.0, 4.3, 4.7 and 5.0 can't be improved.
• All announcements and course materials will be published on Piazza. Please also use Piazza to ask questions, we won't answer questions sent by email.