SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compression

arXiv:2403.07378v1 Announce Type: cross
Abstract: The advancements in Large Language Models (LLMs) have been hindered by their substantial sizes, which necessitate LLM compression methods for practical deployment. Singular Value Decomposition (SVD) offers a promising solution for LLM compression. However, state-of-the-art SVD-based LLM compression methods have two key limitations: truncating smaller singular values may lead to higher compression loss, and the lack of update on the remaining model parameters after SVD truncation. In this work, we propose SVD-LLM, a new SVD-based …

arxiv compression cs.cl cs.lg language language model large language large language model llm singular svd type value