Multimodal Learned Sparse Retrieval with Probabilistic Expansion Control

arXiv:2402.17535v1 Announce Type: cross
Abstract: Learned sparse retrieval (LSR) is a family of neural methods that encode queries and documents into sparse lexical vectors that can be indexed and retrieved efficiently with an inverted index. We explore the application of LSR to the multi-modal domain, with a focus on text-image retrieval. While LSR has seen success in text retrieval, its application in multimodal retrieval remains underexplored. Current approaches like LexLIP and STAIR require complex multi-step training on massive datasets. Our …

abstract application arxiv control cs.cv cs.ir documents domain encode expansion explore family focus image index modal multi-modal multimodal queries retrieval text text-image text-image retrieval type vectors