On Linearizing Structured Data in Encoder-Decoder Language Models: Insights from Text-to-SQL

arXiv:2404.02389v1 Announce Type: new
Abstract: Structured data, prevalent in tables, databases, and knowledge graphs, poses a significant challenge in its representation. With the advent of large language models (LLMs), there has been a shift towards linearization-based methods, which process structured data as sequential token streams, diverging from approaches that explicitly model structure, often as a graph. Crucially, there remains a gap in our understanding of how these linearization-based methods handle structured data, which is inherently non-linear. This work investigates the …

abstract arxiv challenge cs.ai cs.cl data databases decoder encoder encoder-decoder graphs insights knowledge knowledge graphs language language models large language large language models linearization llms process representation shift sql structured data tables text text-to-sql token type