Generation-driven Contrastive Self-training for Zero-shot Text Classification with Instruction-following LLM

arXiv:2304.11872v2 Announce Type: replace
Abstract: The remarkable performance of large language models (LLMs) in zero-shot language understanding has garnered significant attention. However, employing LLMs for large-scale inference or domain-specific fine-tuning requires immense computational resources due to their substantial model size. To overcome these limitations, we introduce a novel method, namely GenCo, which leverages the strong generative power of LLMs to assist in training a smaller and more adaptable language model. In our method, an LLM plays an important role in …

abstract arxiv attention classification computational cs.ai cs.cl domain fine-tuning however inference language language models language understanding large language large language models limitations llm llms novel performance resources scale self-training text text classification training type understanding zero-shot