Previous talks at the SCCS Colloquium

Ludwig Dickmanns: Pose Estimation and Analysis for American Football Videos

SCCS Colloquium |


Strategic analysis of American football is of high importance for live commentary, fans, and coaches and players. For the ladder, mostly in order to prepare properly for the next match. Conventionally, this analysis is done “by hand”. That is, people watch and analyze videos. However, this is a tedious and time-consuming task. In this thesis, I propose an approach to automate the identification of defensive coverage from single frames. For robustness reasons, this approach includes several pre-processing steps. Professional game analysts often use the so-called “all 22” camera angle. “All 22” means that the camera is placed on a high position filming all 22 players on the field. To do tactical analysis, the approach in this thesis uses the “all 22” camera angle, as well. In order to be less susceptible to different lighting and weather conditions, a pose estimation network is used to extract keypoints of the players Most of the time, however, the camera is placed in the middle of the playing field. Therefore, plays on the left side of the camera look different to plays on the right, since camera angles are very different. Thus, this approach employs a projective transformation to get a top-down view of the playing field. Furthermore, the same projective transformation is used on the keypoints to be completely independent of the camera angle and obtain a global 3D model of the playing field and the players in a single frame. Then, this 3D model is used for a sample classification task. For this, a model was trained to distinguish between man and zone coverage. The model is able to predict the correct coverage on the validation and test set data with 85% accuracy. Labels are mispredicted, primarily due to the pose estimation network not being able to detect players. However, accumulating labels over multiple frames of the same play helps to alleviate this problem.

Master's thesis talk. Ludwig is advised by Felix Dietrich.