Divergent Token Metrics: Measuring degradation to prune away LLM components -- and optimize quantization

arXiv:2311.01544v3 Announce Type: replace-cross
Abstract: Large Language Models (LLMs) have reshaped natural language processing with their impressive capabilities. However, their ever-increasing size has raised concerns about their effective deployment and the need for LLM compression. This study introduces the Divergent Token Metrics (DTMs), a novel approach to assessing compressed LLMs, addressing the limitations of traditional perplexity or accuracy measures that fail to accurately reflect text generation quality. DTMs measure token divergences that allow deeper insights into the subtleties of model …

abstract arxiv capabilities components compression concerns cs.cl cs.lg deployment however language language models language processing large language large language models llm llms measuring metrics natural natural language natural language processing novel processing quantization study token type