FLAT: An Optimized Dataflow for Mitigating Attention Bottlenecks. (arXiv:2107.06419v6 [cs.LG] UPDATED)

Attention mechanisms, primarily designed to capture pairwise correlations
between words, have become the backbone of machine learning, expanding beyond
natural language processing into other domains. This growth in adaptation comes
at the cost of prohibitively large memory requirements and computational
complexity, especially at higher number of input elements. This limitation is
due to inherently limited data reuse opportunities and quadratic growth in
memory footprints, leading to severe memory-boundedness and limited scalability
of input elements. This work addresses these challenges by …

arxiv attention dataflow