SemSup-XC.pdf

SemSup-XC is a model for zero-shot and few-shot extreme classification, achieving state-of-the-art performance on three datasets. It uses semantic supervision, representing classes with multiple diverse descriptions, and a novel hybrid matching module that combines semantic and lexical similarity. The model is trained with contrastive learning, sampling a fixed number of negative label descriptions, and establishes a new state-of-the-art with gains of up to 12 precision points on zero-shot and over 10 precision points on one-shot tests.

Key components of SemSup-XC include:

1. Automatic class description collection: A pipeline with precise heuristics to discover high-quality descriptions.
2. Hybrid lexical-semantic similarity model (Hybrid-Match): Combines semantic similarity of sentences with relaxed lexical-matching between all token pairs.
3. Contrastive learning objective: Samples a fixed number of negative label descriptions, improving computation speed by up to 99% compared to previous work.

SemSup-XC outperforms baselines and larger unsupervised language models, and ablation studies highlight the importance of the hybrid matching module and automatically collected class descriptions. The model predicts diverse correct classes, applicable to the instance at times, whereas other models either predict incorrect classes or suffer a mode collapse.