Efficient Few-shot Learning for Multi-label Classification of Scientific Documents with Many Classes
Scientific document classification is a critical task and often involves many classes. However, collecting human-labeled data for many classes is expensive and usually leads to label-scarce scenarios. Moreover, recent work has shown that sentence embedding model fine-tuning for few-shot classification is efficient, robust, and effective. In this work, we propose FusionSent (Fusion-based Senence Embedding Fine-tuning), an efficient and prompt-free approach for few-shot classification of scientific documents with many classes. FusionSent uses available training examples and their respective label texts to contrastively fine-tune two different sentence embedding models. Afterward, the parameters of both fine-tuned models are fused to combine the complementary knowledge from the separate fine-tuning steps into a single model. Finally, the resulting sentence embedding model is frozen to embed the training instances, which are then used as input features to train a classification head. Our experiments show that FusionSent significantly outperforms strong baselines by an average of 6.0 F1 points across multiple scientific document classification datasets. In addition, we introduce a new dataset for multi-label classification of scientific documents, which contains 203,961 scientific articles and 130 classes from the arXiv category taxonomy. Code and data are available at https://github.com/sebischair/FusionSent.
| Attribute | Value |
|---|---|
| Address | Trento, Italy |
| Authors | Tim Schopf , Alexander Blatzheim , Nektarios Machner , Prof. Dr. Florian Matthes |
| Citation | @inproceedings{schopf-etal-2024-efficient, title = "Efficient Few-shot Learning for Multi-label Classification of Scientific Documents with Many Classes", author = "Schopf, Tim and Blatzheim, Alexander and Machner, Nektarios and Matthes, Florian", editor = "Abbas, Mourad and Freihat, Abed Alhakim", booktitle = "Proceedings of the 7th International Conference on Natural Language and Speech Processing (ICNLSP 2024)", month = oct, year = "2024", address = "Trento", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2024.icnlsp-1.21", pages = "186--198", } |
| Key | Sc24b |
| Research project | Natural Language Processing Knowledge Graph (NLP-KG) |
| Title | Efficient Few-shot Learning for Multi-label Classification of Scientific Documents with Many Classes |
| Type of publication | Conference |
| Year | 2024 |
| Publication URL | https://aclanthology.org/2024.icnlsp-1.21 |
| Project | Natural Language Processing Knowledge Graph (NLP-KG) |
| Acronym | |
| Team members |