UniBind: LLM-Augmented Unified and Balanced Representation Space to Bind Them All

arXiv:2403.12532v1 Announce Type: new
Abstract: We present UniBind, a flexible and efficient approach that learns a unified representation space for seven diverse modalities -- images, text, audio, point cloud, thermal, video, and event data. Existing works, eg., ImageBind, treat the image as the central modality and build an image-centered representation space; however, the space may be sub-optimal as it leads to an unbalanced representation space among all modalities. Moreover, the category names are directly used to extract text embeddings for …

abstract arxiv audio build cloud cs.cv data diverse event image imagebind images llm representation space text them type video