Skip to content
Technical University of Munich

AI Benchmarking on IBM Power Hardware

Thesis (MA)

Advisor(s): Simon Fuchs, Maximilian Niedermeier, Noah Kim

Context

With Machine Learning (ML) and Artificial Intelligence (AI) algorithms becoming increasingly common, the need for considerable computing center resources has increased dramatically. This drastic increase in the need for computing resources is aligned with an inefficient use, primarily due to the significantly varying consumption of such resources and the novelty of many AI technologies. 

In recent years, Large Language Models (LLMs) have become popular for their outstanding results in all Natural Language Processing (NLP) fields. Generative AI chatbots like ChatGPT would not be possible without Large Language Transformer Models. However, LLMs consume enormous computing power, significantly more than all their predecessor technologies. With this, an efficient use of their resources and an evaluation of their overall consumption becomes necessary. Scientifically approaching the evaluation and measurement of IT is the primary task of benchmarking. 

The Forschungsgruppe Wittges, chair of Information Systems and Business Process Management (I17), runs a small computing center using IBM Power hardware to host multiple enterprise software technologies and has a significant research legacy in benchmarking enterprise software on IBM Power servers. Now, we plan to extend this research to the field of benchmarking AI enterprise software technology.   

Against this context, the announced master's thesis aims for three goals

  1. Identify the current state-of-the-art AI benchmarking in the scientific literature.
  2. Conduct prototypical benchmarking of enterprise-relevant LLMs on our IBM Power Servers.
  3. Develop an optimization plan for more effective resource usage of small LLMs in an educational context.

Tasks

  • Review the scientific literature in the specific field
  • Review the chair’s previous work in the field
  • Invest time and effort to understand the use case
  • Communicate with other students running ML and AI projects at the chair
  • Do some benchmarking experiments
  • Develop an optimization plan for our specific use case

Requirements

  • High degree of autonomy and responsibility
  • Interest in research on computing centers and hardware benchmarking
  • Interest in Natural Language Processing
  • First experiences with Linux
  • First experiences with NLP and LLMs in Python
  • Good language skills in German and English

What we offer

  • An opportunity to write your master thesis
  • Practical ML experience in an IBM related project
  • Insights into everyday research at a chair
  • A cool team :D to work in

Further Information

If you have further questions, please do not hesitate to contact me directly (s.t.fuchs(at)tum.de). Please send your application including a current transcript of records and your CV to s.t.fuchs(at)tum.de.

To top