AI21 Labs’ Augmented Frozen Language Models Challenge Conventional Fine-Tuning Approaches Without Sacrificing Versatility

In the new paper Standing on the Shoulders of Giant Frozen Language Models, AI21 Labs researchers propose three novel methods for learning small neural modules that specialize a frozen language model to different tasks. Their compute-saving approach outperforms conventional frozen model methods and challenges fine-tuning performance without sacrificing model versatility.

ai ai21 labs artificial intelligence challenge deep-neural-networks fine-tuning labs language language model language models machine learning machine learning & data science ml research technology