A Probabilistic Perspective on Unlearning and Alignment for Large Language Models

This page links to additional material for our paper

A Probabilistic Perspective on Unlearning and Alignment for Large Language Models
Yan Scholten, Stephan Günnemann, Leo Schwinn
International Conference on Learning Representations, ICLR 2025 (Oral)

Links

[PDF | LLM framework (Code)| Confidence bounds (Code) | Slides | Poster]

Abstract

Comprehensive evaluation of Large Language Models (LLMs) is an open research problem. Existing evaluations rely on deterministic point estimates generated via greedy decoding. However, we find that deterministic evaluations fail to capture the whole output distribution of a model, yielding inaccurate estimations of model capabilities. This is particularly problematic in critical contexts such as unlearning and alignment, where precise model evaluations are crucial. To remedy this, we introduce the first formal probabilistic evaluation framework in LLMs. Namely, we derive novel metrics with high-probability guarantees concerning the output distribution of a model. Our metrics are application-independent and allow practitioners to make more reliable estimates about model capabilities before deployment. Through a case study focused on unlearning, we reveal that deterministic evaluations falsely indicate successful unlearning, whereas our probabilistic evaluations demonstrate that most if not all of the supposedly unlearned information remains accessible in these models. Additionally, we propose a novel unlearning loss based on entropy optimization and adaptive temperature scaling, which significantly improves unlearning in probabilistic settings on recent benchmarks. Our proposed shift from point estimates to probabilistic evaluations of output distributions represents an important step toward comprehensive evaluations of LLMs.

Cite

@inproceedings{scholten2025probabilistic,

     title={A Probabilistic Perspective on Unlearning and Alignment for Large Language Models},

     author={Yan Scholten and Stephan G{\"u}nnemann and Leo Schwinn},

     booktitle={The Thirteenth International Conference on Learning Representations},

     year={2025},

     url={https://openreview.net/forum?id=51WraMid8K}

}