Making Pre-trained Language Models Great on Tabular Prediction

arXiv:2403.01841v1 Announce Type: cross
Abstract: The transferability of deep neural networks (DNNs) has made significant progress in image and language processing. However, due to the heterogeneity among tables, such DNN bonus is still far from being well exploited on tabular data prediction (e.g., regression or classification tasks). Condensing knowledge from diverse domains, language models (LMs) possess the capability to comprehend feature names from various tables, potentially serving as versatile learners in transferring knowledge across distinct tables and diverse prediction tasks, …

abstract arxiv bonus classification cs.cl cs.lg data diverse dnn domains image knowledge language language models language processing making networks neural networks prediction processing progress regression tables tabular tabular data tasks type