Lexically Faithful Soft Compression for RAG
Retrieval-Augmented Generation (RAG) improves question answering by grounding language models in external documents, but long retrieved contexts increase inference cost, latency, memory usage, and dependence on large cloud infrastructure. Prompt compression methods, especially soft compression, aim to address this by representing documents as compact dense embeddings. However, dense representations are known to lose exact lexical information such as entity names, identifiers, numbers, dates, and rare domain-specific terms.
This thesis investigates whether soft compression exhibits similar lexical information loss in RAG settings. It proposes benchmarks for evaluating lexical faithfulness, including augmented QA datasets with substituted entities and experiments on entity-centric question answering datasets. The thesis further explores entity-aware soft compression mechanisms, such as lexical side-channels or auxiliary entity-preservation objectives, to improve soft compression while keeping its efficiency benefits. The expected contribution is a diagnostic benchmark for evaluating lexical faithfulness of prompt compression methods and an initial design direction for lexically faithful soft compression in RAG.
Research Questions:
RQ1: To what extent do prompt compression methods preserve exact lexical information, such as entity names, identifiers, and rare terms, in retrieval-augmented question answering?
RQ2: How can prompt compression methods be benchmarked for their ability to preserve entity-specific and lexical information?
RQ3: Can methods such as lexical side-channel or introducing auxiliary training objectives improve the lexical faithfulness of soft compression with minimal additional compute cost?
| Attribute | Value |
|---|---|
| Title (de) | Lexically Faithful Soft Compression for RAG |
| Title (en) | Lexically Faithful Soft Compression for RAG |
| Project | |
| Type | Master's Thesis |
| Status | started |
| Student | Burak Ayaz |
| Advisor | Ibrahim Ebrar Yurt |
| Supervisor | Prof. Dr. Florian Matthes |
| Start Date | 01.07.2026 |
| Sebis Contributor Agreement signed on | 01.07.2026 |
| Checklist filled | Yes |
| Submission date | 04.01.2027 |