Language Models As Semantic Indexers

arXiv:2310.07815v2 Announce Type: replace-cross
Abstract: Semantic identifier (ID) is an important concept in information retrieval that aims to preserve the semantics of objects such as documents and items inside their IDs. Previous studies typically adopt a two-stage pipeline to learn semantic IDs by first procuring embeddings using off-the-shelf text encoders and then deriving IDs based on the embeddings. However, each step introduces potential information loss, and there is usually an inherent mismatch between the distribution of embeddings within the latent …

arxiv cs.cl cs.ir cs.lg language language models semantic type