Optimizing LLM Queries in Relational Workloads

arXiv:2403.05821v1 Announce Type: new
Abstract: Analytical database providers (e.g., Redshift, Databricks, BigQuery) have rapidly added support for invoking Large Language Models (LLMs) through native user-defined functions (UDFs) to help users perform natural language tasks, such as classification, entity extraction, and translation, inside analytical workloads. For instance, an analyst might want to extract customer sentiments on millions of product reviews. However, LLM inference is highly expensive in both computational and economic terms: for example, an NVIDIA L4 GPU running Llama2-7B can …

abstract analyst analytical database analytical workloads arxiv bigquery classification cs.db cs.lg customer database databricks extract extraction functions inside instance language language models large language large language models llm llms natural natural language queries redshift relational support tasks through translation type workloads