LLMs for Legal Subsumption in German Employment Contracts
Legal work, characterized by its text-heavy and resource-intensive nature, presents unique challenges and opportunities for NLP research. While data-driven approaches have advanced the field, their lack of interpretability and trustworthiness limits their applicability in dynamic legal environments. To address these issues, we collaborated with legal experts to extend an existing dataset and explored the use of Large Language Models (LLMs) and in-context learning to evaluate the legality of clauses in German employment contracts. Our work evaluates the ability of different LLMs to classify clauses as "valid," "unfair," or "void" under three legal context variants: no legal context, full-text sources of laws and court rulings, and distilled versions of these (referred to as examination guidelines). Results show that full-text sources moderately improve performance, while examination guidelines significantly enhance recall for void clauses and weighted F1-Score, reaching 80\%. Despite these advancements, LLMs' performance when using full-text sources remains substantially below that of human lawyers. We contribute an extended dataset, including examination guidelines, referenced legal sources, and corresponding annotations, alongside our code and all log files. Our findings highlight the potential of LLMs to assist lawyers in contract legality review while also underscoring the limitations of the methods presented.
Data: https://github.com/sebischair/Employment-Contract-Clauses-German
Published @ ICAIL 25
| Attribute | Value |
|---|---|
| Address | Chicago, USA |
| Authors | Oliver Wardas |
| Citation | Wardas, O; Matthes F.: LLMs for Legal Subsumption in German Employment Contracts. In Proceedings of the 20th International Conference on Artificial Intelligence and Law (ICAIL25). Chicago, USA. 2025. |
| Key | Wa25b |
| Research project | AI-Assisted Legal Analysis and Correction of German Employment Contracts |
| Title | LLMs for Legal Subsumption in German Employment Contracts |
| Type of publication | Conference |
| Year | 2025 |
| Acronym | ICAIL |
| Project | |
| Publication URL | |
| Team members |