Embedding Hallucination for Few-Shot Language Fine-tuning. (arXiv:2205.01307v1 [cs.CL])

Few-shot language learners adapt knowledge from a pre-trained model to
recognize novel classes from a few-labeled sentences. In such settings,
fine-tuning a pre-trained language model can cause severe over-fitting. In this
paper, we propose an Embedding Hallucination (EmbedHalluc) method, which
generates auxiliary embedding-label pairs to expand the fine-tuning dataset.
The hallucinator is trained by playing an adversarial game with the
discriminator, such that the hallucinated embedding is indiscriminative to the
real ones in the fine-tuning dataset. By training with the …

arxiv embedding fine-tuning language