SAFT: Structure-Aware Fine-Tuning of Large Language Models for AMR-to-Text Generation
Abstract
Large Language Models (LLMs) are increasingly applied to tasks involving structured inputs such as semantic graphs, yet adapting them to such inputs remains non-trivial. Common approaches either linearize graphs, discarding structural information, or rely on specialized architectures that are not directly compatible with standard pretrained LLMs. We present SAFT, a structure-aware fine-tuning method that augments LLMs with graph positional encodings derived from the magnetic Laplacian of the input graph. These encodings are projected into the LLM embedding space, introducing relational inductive bias without modifying the model architecture. While SAFT is conceptually applicable to directed graph inputs with node--token alignment, we evaluate it only on the task of generating natural language text from an input AMR (Abstract Meaning Representation) graph. AMRs are directed graphs encoding predicate-argument semantics of natural language sentences, and AMR-to-text generation requires models to integrate both linguistic fluency and structural faithfulness, making it a demanding evaluation setting. We show that SAFT consistently improves or matches standard fine-tuning across six evaluated decoder-only LLMs from 0.5B to 3B parameters, with gains that increase with AMR structural complexity on both sentence-level graphs of increasing depth and document-level graphs of increasing size. These results demonstrate that structural encoding provides a reliable inductive bias for AMR-to-text fine-tuning within the evaluated model range.
Links
Citation
SAFT: Structure-Aware Fine-Tuning of Large Language Models for AMR-to-Text Generation
Rafiq Kamel*, Filippo Guerranti*, Simon Geisler, Stephan Günnemann
TMLR, 2026
If you use this work in your research, please cite this paper:
@article{kamel2026saft,
title={{SAFT}: Structure-Aware Fine-Tuning of Large Language Models for {AMR}-to-Text Generation},
author={Anonymous},
journal={Transactions on Machine Learning Research},
year={2026},
url={https://openreview.net/forum?id=QZoUMyzYDB}
}