Controlling Vision-Language Models for Multi-Task Image Restoration

arXiv:2310.01018v2 Announce Type: replace
Abstract: Vision-language models such as CLIP have shown great impact on diverse downstream tasks for zero-shot or label-free predictions. However, when it comes to low-level vision such as image restoration their performance deteriorates dramatically due to corrupted inputs. In this paper, we present a degradation-aware vision-language model (DA-CLIP) to better transfer pretrained vision-language models to low-level vision tasks as a multi-task framework for image restoration. More specifically, DA-CLIP trains an additional controller that adapts the fixed …

arxiv cs.cv image image restoration language language models type vision vision-language models