CodeS: Towards Building Open-source Language Models for Text-to-SQL

arXiv:2402.16347v1 Announce Type: new
Abstract: Language models have shown promising performance on the task of translating natural language questions into SQL queries (Text-to-SQL). However, most of the state-of-the-art (SOTA) approaches rely on powerful yet closed-source large language models (LLMs), such as ChatGPT and GPT-4, which may have the limitations of unclear model architectures, data privacy risks, and expensive inference overheads. To address the limitations, we introduce CodeS, a series of pre-trained language models with parameters ranging from 1B to 15B, …

abstract art arxiv building chatgpt cs.cl cs.db gpt gpt-4 language language models large language large language models limitations llms natural natural language performance queries questions sota sql sql queries state text text-to-sql type