Learning with SASQuaTCh: a Novel Variational Quantum Transformer Architecture with Kernel-Based Self-Attention

arXiv:2403.14753v1 Announce Type: cross
Abstract: The widely popular transformer network popularized by the generative pre-trained transformer (GPT) has a large field of applicability, including predicting text and images, classification, and even predicting solutions to the dynamics of physical systems. In the latter context, the continuous analog of the self-attention mechanism at the heart of transformer networks has been applied to learning the solutions of partial differential equations and reveals a convolution kernel nature that can be exploited by the Fourier …

abstract analog architecture arxiv attention classification context continuous cs.lg dynamics generative generative pre-trained transformer gpt images kernel network novel popular quant-ph quantum self-attention solutions systems text transformer transformer architecture transformer network type