TuneTables: Context Optimization for Scalable Prior-Data Fitted Networks

arXiv:2402.11137v1 Announce Type: new
Abstract: While tabular classification has traditionally relied on from-scratch training, a recent breakthrough called prior-data fitted networks (PFNs) challenges this approach. Similar to large language models, PFNs make use of pretraining and in-context learning to achieve strong performance on new tasks in a single forward pass. However, current PFNs have limitations that prohibit their widespread adoption. Notably, TabPFN achieves very strong performance on small tabular datasets but is not designed to make predictions for datasets of …

abstract arxiv challenges classification context cs.lg current data from-scratch in-context learning language language models large language large language models networks optimization performance pretraining prior scalable scratch tabular tasks training type