Pretrained deep models outperform GBDTs in Learning-To-Rank under label scarcity

arXiv:2308.00177v3 Announce Type: replace
Abstract: On tabular data, a significant body of literature has shown that current deep learning (DL) models perform at best similarly to Gradient Boosted Decision Trees (GBDTs), while significantly underperforming them on outlier data. We identify a natural tabular data setting where DL models can outperform GBDTs: tabular Learning-to-Rank (LTR) under label scarcity. Tabular LTR applications, including search and recommendation, often have an abundance of unlabeled data, and scarce labeled data. We show that DL rankers …

abstract arxiv cs.ai cs.lg current data decision decision trees deep learning gradient identify learning-to-rank literature natural outlier tabular tabular data them trees type