Evaluating the Data Model Robustness of Text-to-SQL Systems Based on Real User Queries

Text-to-SQL systems (also known as NL-to-SQL systems) have become an increasingly popular solution for bridging the gap between user capabilities and SQL-based data access. These systems translate user requests in natural language to valid SQL statements for a specific database. Recent Text-to-SQL systems have benefited from the rapid improvement of transformer-based language models. However, while Text-to-SQL systems that incorporate such models continuously reach new high scores on -- often synthetic -- benchmark datasets, a systematic exploration of their robustness towards …

become capabilities cs.ai cs.cl cs.db data data access database data model gap language model robustness natural natural language popular robustness solution sql systems text text-to-sql translate