HiP: Hierarchical Perceiver. (arXiv:2202.10890v2 [cs.CV] UPDATED)

General perception systems such as Perceivers can process arbitrary
modalities in any combination and are able to handle up to a few hundred
thousand inputs. They achieve this generality by using exclusively global
attention operations. This however hinders them from scaling up to the inputs
sizes required to process raw high-resolution images or video. In this paper,
we show that some degree of locality can be introduced back into these models,
greatly improving their efficiency while preserving their generality. To …

arxiv hierarchical perceiver