OpenBA-V2: Reaching 77.3% High Compression Ratio with Fast Multi-Stage Pruning

arXiv:2405.05957v1 Announce Type: new
Abstract: Large Language Models (LLMs) have played an important role in many fields due to their powerful capabilities.However, their massive number of parameters leads to high deployment requirements and incurs significant inference costs, which impedes their practical applications. Training smaller models is an effective way to address this problem. Therefore, we introduce OpenBA-V2, a 3.4B model derived from multi-stage compression and continual pre-training from the original 15B OpenBA model. OpenBA-V2 utilizes more data, more flexible training …

abstract applications arxiv capabilities compression costs cs.cl deployment fields however inference inference costs language language models large language large language models leads llms massive parameters practical pruning requirements role stage training type