Evaluating Unsupervised Text Classification - Zero-shot and Similarity-based Approaches
Text classification of unseen classes is a challenging Natural Language Processing task and is mainly attempted using two different types of approaches. Similarity-based approaches attempt to classify instances based on similarities between text document representations and class description representations. Zero-shot text classification approaches aim to generalize knowledge gained from a training task by assigning appropriate labels of unknown classes to text documents. Although existing studies have already investigated individual approaches to these categories, the experiments in literature do not provide a consistent comparison. This paper addresses this gap by conducting a systematic evaluation of different similarity-based and zero-shot approaches for text classification of unseen classes. Different state-of-the-art approaches are benchmarked on four text classification datasets, including a new dataset from the medical domain. Additionally, novel SimCSE and SBERT-based baselines are proposed, as other baselines used in existing work yield weak classification results and are easily outperformed. Finally, the novel similarity-based Lbl2TransformerVec approach is presented, which outperforms previous state-of-the-art approaches in unsupervised text classification. Our experiments show that similarity-based approaches significantly outperform zero-shot approaches in most cases. Additionally, using SimCSE or SBERT embeddings instead of simpler text representations increases similarity-based classification results even further.
Blog articles about this paper:
| Attribute | Value |
|---|---|
| Address | Bangkok, Thailand |
| Authors | Tim Schopf , Dr. Daniel Braun , Prof. Dr. Florian Matthes |
| Citation | @inproceedings{schopf_etal_nlpir22, author = {Schopf, Tim and Braun, Daniel and Matthes, Florian}, title = {Evaluating Unsupervised Text Classification: Zero-shot and Similarity-based Approaches}, year = {2022}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, booktitle = {2022 6th International Conference on Natural Language Processing and Information Retrieval (NLPIR)}, keywords = {Natural Language Processing, Unsupervised Text Classification, Zero-shot Text Classification}, location = {Bangkok, Thailand}, series = {NLPIR 2022} } |
| Key | Sc23b |
| Research project | |
| Title | Evaluating Unsupervised Text Classification - Zero-shot and Similarity-based Approaches |
| Type of publication | Conference |
| Year | 2023 |
| Publication URL | https://doi.org/10.1145/3582768.3582795 |
| Acronym | |
| Project | |
| Team members |