Towards Unified Task Embeddings Across Multiple Models: Bridging the Gap for Prompt-Based Large Language Models and Beyond

arXiv:2402.14522v1 Announce Type: cross
Abstract: Task embedding, a meta-learning technique that captures task-specific information, has become prevalent, especially in areas such as multi-task learning, model editing, and interpretability. However, it faces challenges with the emergence of prompt-guided Large Language Models (LLMs) operating in a gradientfree manner. Existing task embedding methods rely on fine-tuned, task-specific language models, which hinders the adaptability of task embeddings across diverse models, especially prompt-based LLMs. To unleash the power of task embedding in the era of …

abstract arxiv become beyond challenges cs.cl cs.lg editing embedding embeddings emergence gap information interpretability language language models large language large language models llms meta meta-learning multiple multi-task learning prompt type