Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs. (arXiv:2305.03111v1 [cs.CL])

Text-to-SQL parsing, which aims at converting natural language instructions
into executable SQLs, has gained increasing attention in recent years. In
particular, Codex and ChatGPT have shown impressive results in this task.
However, most of the prevalent benchmarks, i.e., Spider, and WikiSQL, focus on
database schema with few rows of database contents leaving the gap between
academic study and real-world applications. To mitigate this gap, we present
Bird, a big benchmark for large-scale database grounded in text-to-SQL tasks,
containing 12,751 pairs …

arxiv attention benchmarks big chatgpt codex database focus language llm natural natural language parsing scale serve sql text