LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression

arXiv:2403.12968v1 Announce Type: cross
Abstract: This paper focuses on task-agnostic prompt compression for better generalizability and efficiency. Considering the redundancy in natural language, existing approaches compress prompts by removing tokens or lexical units according to their information entropy obtained from a causal language model such as LLaMa-7B. The challenge is that information entropy may be a suboptimal compression metric: (i) it only leverages unidirectional context and may fail to capture all essential information needed for prompt compression; (ii) it is …

abstract arxiv causal challenge compression cs.cl cs.lg data distillation efficiency entropy information language language model llama llama-7b natural natural language paper prompt prompts redundancy tokens type units