Variance-reduced Zeroth-Order Methods for Fine-Tuning Language Models

arXiv:2404.08080v1 Announce Type: new
Abstract: Fine-tuning language models (LMs) has demonstrated success in a wide array of downstream tasks. However, as LMs are scaled up, the memory requirements for backpropagation become prohibitively high. Zeroth-order (ZO) optimization methods can leverage memory-efficient forward passes to estimate gradients. More recently, MeZO, an adaptation of ZO-SGD, has been shown to consistently outperform zero-shot and in-context learning when combined with suitable task prompts. In this work, we couple ZO methods with variance reduction techniques to …

abstract array arxiv backpropagation become cs.ai cs.cl cs.lg fine-tuning however language language models lms math.oc memory optimization requirements success tasks type variance