Transformer models: an introduction and catalog

arXiv:2302.07730v4 Announce Type: replace
Abstract: In the past few years we have seen the meteoric appearance of dozens of foundation models of the Transformer family, all of which have memorable and sometimes funny, but not self-explanatory, names. The goal of this paper is to offer a somewhat comprehensive but simple catalog and classification of the most popular Transformer models. The paper also includes an introduction to the most important aspects and innovations in Transformer models. Our catalog will include models …

abstract arxiv classification cs.cl family foundation funny introduction paper simple transformer transformer models type