RASAT: Integrating Relational Structures into Pretrained Seq2Seq Model for Text-to-SQL. (arXiv:2205.06983v2 [cs.CL] UPDATED)

Relational structures such as schema linking and schema encoding have been
validated as a key component to qualitatively translating natural language into
SQL queries. However, introducing these structural relations comes with prices:
they often result in a specialized model structure, which largely prohibits
using large pretrained models in text-to-SQL. To address this problem, we
propose RASAT: a Transformer seq2seq architecture augmented with relation-aware
self-attention that could leverage a variety of relational structures while
inheriting the pretrained parameters from the T5 …

arxiv seq2seq sql text