Smart-Infinity: Fast Large Language Model Training using Near-Storage Processing on a Real System

arXiv:2403.06664v1 Announce Type: cross
Abstract: The recent huge advance of Large Language Models (LLMs) is mainly driven by the increase in the number of parameters. This has led to substantial memory capacity requirements, necessitating the use of dozens of GPUs just to meet the capacity. One popular solution to this is storage-offloaded training, which uses host memory and storage as an extended memory hierarchy. However, this obviously comes at the cost of storage bandwidth bottleneck because storage devices have orders …

abstract advance arxiv capacity cs.ar cs.lg gpus language language model language models language model training large language large language model large language models llms memory near parameters popular processing requirements smart storage training type