Compositional LLMs - Paper from Deepmind introduces CALM

Abstract:
Foundational models with billions of parameters which have been trained on large corpora of data have demonstrated non-trivial skills in a variety of domains. However, due to their monolithic structure, it is challenging and expensive to augment them or impart new skills. On the other hand, due to their adaptation abilities, several new instances of these models are being trained towards new domains and tasks. In this work, we study the problem of efficient and practical composition of existing …

abstract data deepmind domains foundational models llms machinelearningnews paper parameters skills them