CHORUS: Foundation Models for Unified Data Discovery and Exploration

arXiv:2306.09610v3 Announce Type: replace-cross
Abstract: We apply foundation models to data discovery and exploration tasks. Foundation models include large language models (LLMs) that show promising performance on a range of diverse tasks unrelated to their training. We show that these models are highly applicable to the data discovery and data exploration domain. When carefully used, they have superior capability on three representative tasks: table-class detection, column-type annotation and join-column prediction. On all three tasks, we show that a foundation-model-based approach …

abstract apply arxiv cs.db cs.lg data data discovery data exploration discovery diverse exploration foundation language language models large language large language models llms performance show tasks training type unified data