The GatedTabTransformer. An enhanced deep learning architecture for tabular modeling. (arXiv:2201.00199v1 [cs.LG])

There is an increasing interest in the application of deep learning
architectures to tabular data. One of the state-of-the-art solutions is
TabTransformer which incorporates an attention mechanism to better track
relationships between categorical features and then makes use of a standard MLP
to output its final logits. In this paper we propose multiple modifications to
the original TabTransformer performing better on binary classification tasks
for three separate datasets with more than 1% AUROC gains. Inspired by gated
MLP, linear projections …

architecture arxiv deep learning learning modeling