Transformers, Contextualism, and Polysemy

arXiv:2404.09577v1 Announce Type: new
Abstract: The transformer architecture, introduced by Vaswani et al. (2017), is at the heart of the remarkable recent progress in the development of language models, including famous chatbots such as Chat-gpt and Bard. In this paper, I argue that we an extract from the way the transformer architecture works a picture of the relationship between context and meaning. I call this the transformer picture, and I argue that it is a novel with regard to two …

abstract architecture arxiv bard chat chatbots cs.ai cs.cl development extract gpt language language models paper progress the way transformer transformer architecture transformers type