Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Tasks. (arXiv:2206.08916v1 [cs.CV])

We propose Unified-IO, a model that performs a large variety of AI tasks
spanning classical computer vision tasks, including pose estimation, object
detection, depth estimation and image generation, vision-and-language tasks
such as region captioning and referring expression comprehension, to natural
language processing tasks such as question answering and paraphrasing.
Developing a single unified model for such a large variety of tasks poses
unique challenges due to the heterogeneous inputs and outputs pertaining to
each task, including RGB images, per-pixel maps, …

arxiv cv io language unified model vision