When Reasoning Becomes Overthinking: Declining Legal Classification Performance in Reasoning LLMs

---

The integration of reasoning capabilities into Large Language Models (LLMs) has yielded significant performance gains in closed environments such as mathematics and programming, offering new potential for complex problem-solving. However, the efficacy of these models in normative domains, such as the legal field, remains underexplored. To address this gap, we benchmarked flagship reasoning models (GPT-5, Gemini-2.5-pro) and their non-reasoning counterparts (GPT-4.1, Gemini-2.5-flash) on the German Employment Contract dataset. Our work evaluates the impact of varying reasoning effort on clause legality classification accuracy. Results reveal a counter-intuitive "overthinking" phenomenon: increased reasoning effort leads to a decline in performance, driven by a hyper-critical expansion of ambiguous legal terms. Contradicting the general superiority of reasoning models in other domains, the non-reasoning model GPT-4.1 achieved the highest macro-averaged F1 score of 0.60, significantly outperforming the flagship reasoning models GPT-5 and Gemini-2.5-pro. We demonstrate that this decline stems from an excessively critical handling of ambiguity rather than deficiencies in instruction following or legal understanding. These findings highlight potential limitations in current reasoning training paradigms and underscore the critical necessity of task-specific benchmarking over reliance on general benchmarks and leaderboards.

 

Data: https://github.com/sebischair/Employment-Contract-Clauses-German

 

Published @ ICAIL 26

---

Attribute	Value
Address	Singapore
Authors	Oliver Wardas
Citation
Key	Wa26a
Research project	AI-Assisted Legal Analysis and Correction of German Employment Contracts
Title	When Reasoning Becomes Overthinking: Declining Legal Classification Performance in Reasoning LLMs
Type of publication	Conference
Year	2026
Acronym	ICAIL
Project
Publication URL
Team members