Say More with Less: Understanding Prompt Learning Behaviors through Gist Compression

arXiv:2402.16058v1 Announce Type: new
Abstract: Large language models (LLMs) require lengthy prompts as the input context to produce output aligned with user intentions, a process that incurs extra costs during inference. In this paper, we propose the Gist COnditioned deCOding (Gist-COCO) model, introducing a novel method for compressing prompts which also can assist the prompt interpretation and engineering. Gist-COCO employs an encoder-decoder based language model and then incorporates an additional encoder as a plugin module to compress prompts with inputs …

arxiv compression cs.cl prompt prompt learning through type understanding