ZipIt! Merging Models from Different Tasks without Training

arXiv:2305.03053v3 Announce Type: replace-cross
Abstract: Typical deep visual recognition models are capable of performing the one task they were trained on. In this paper, we tackle the extremely difficult problem of combining distinct models with different initializations, each solving a separate task, into one multi-task model without any additional training. Prior work in model merging permutes one model to the space of the other then averages them together. While this works for models trained on the same task, we find …

abstract arxiv cs.cv cs.lg merging paper prior recognition tasks training type visual work