In-Context Translation: Towards Unifying Image Recognition, Processing, and Generation

arXiv:2404.09633v1 Announce Type: new
Abstract: We propose In-Context Translation (ICT), a general learning framework to unify visual recognition (e.g., semantic segmentation), low-level image processing (e.g., denoising), and conditional image generation (e.g., edge-to-image synthesis). Thanks to unification, ICT significantly reduces the inherent inductive bias that comes with designing models for specific tasks, and it maximizes mutual enhancement across similar tasks. However, the unification across a large number of tasks is non-trivial due to various data formats and training pipelines. To this …

abstract arxiv bias context cs.cv denoising designing edge framework general ict image image generation image processing image recognition inductive low processing recognition segmentation semantic specific tasks synthesis tasks translation type unification visual