AffineQuant: Affine Transformation Quantization for Large Language Models

arXiv:2403.12544v1 Announce Type: new
Abstract: The significant resource requirements associated with Large-scale Language Models (LLMs) have generated considerable interest in the development of techniques aimed at compressing and accelerating neural networks. Among these techniques, Post-Training Quantization (PTQ) has emerged as a subject of considerable interest due to its noteworthy compression efficiency and cost-effectiveness in the context of training. Existing PTQ methods for LLMs limit the optimization scope to scaling transformations between pre- and post-quantization weights. In this paper, we advocate …

abstract accelerating neural networks arxiv compression cs.lg development efficiency generated language language models large language large language models llms networks neural networks quantization requirements scale training transformation type