Bidirectional Long-Range Parser for Sequential Data Understanding

arXiv:2404.05210v1 Announce Type: cross
Abstract: The transformer is a powerful data modelling framework responsible for remarkable performance on a wide range of tasks. However, they are limited in terms of scalability as it is suboptimal and inefficient to process long-sequence data. To this purpose we introduce BLRP (Bidirectional Long-Range Parser), a novel and versatile attention mechanism designed to increase performance and efficiency on long-sequence tasks. It leverages short and long range heuristics in the form of a local sliding window …

abstract arxiv cs.cl cs.cv cs.lg data framework however modelling novel performance process responsible scalability tasks terms transformer type understanding