STaRK: Benchmarking LLM Retrieval on Textual and Relational Knowledge Bases

arXiv:2404.13207v1 Announce Type: cross
Abstract: Answering real-world user queries, such as product search, often requires accurate retrieval of information from semi-structured knowledge bases or databases that involve blend of unstructured (e.g., textual descriptions of products) and structured (e.g., entity relations of products) information. However, previous works have mostly studied textual and relational retrieval tasks as separate topics. To address the gap, we develop STARK, a large-scale Semi-structure retrieval benchmark on Textual and Relational Knowledge Bases. We design a novel pipeline …

abstract arxiv benchmarking blend cs.ir cs.lg databases however information knowledge llm product products product search queries relational relations retrieval search textual type unstructured world