Recycling finetuned models to pretrain: on loss spaces, fusing and evolving pretraining

This work will discuss the recent advancement in recycling finetuned model. Harnessing the data and computation invested in one or more models to collaboratively improve the pre-trained model they originated form. The work will also touch on our initial understandings of how and why fusing several models by weight averaging works. - Leshem Choshen/IBM Research

computation data discuss ibm ibm research loss recycling research spaces work