Enhancing Inference Efficiency of Large Language Models: Investigating Optimization Strategies and Architectural Innovations

arXiv:2404.05741v1 Announce Type: new
Abstract: Large Language Models are growing in size, and we expect them to continue to do so, as larger models train quicker. However, this increase in size will severely impact inference costs. Therefore model compression is important, to retain the performance of larger models, but with a reduced cost of running them. In this thesis we explore the methods of model compression, and we empirically demonstrate that the simple method of skipping latter attention sublayers in …

abstract arxiv compression costs cs.ai cs.cl cs.lg cs.pf efficiency expect however impact inference inference costs innovations language language models large language large language models larger models optimization performance strategies them train type will