EyeFormer: Predicting Personalized Scanpaths with Transformer-Guided Reinforcement Learning

arXiv:2404.10163v1 Announce Type: cross
Abstract: From a visual perception perspective, modern graphical user interfaces (GUIs) comprise a complex graphics-rich two-dimensional visuospatial arrangement of text, images, and interactive objects such as buttons and menus. While existing models can accurately predict regions and objects that are likely to attract attention ``on average'', so far there is no scanpath model capable of predicting scanpaths for an individual. To close this gap, we introduce EyeFormer, which leverages a Transformer architecture as a policy network …

abstract arxiv attention cs.ai cs.cv cs.hc cs.lg graphics images interactive interfaces modern objects perception personalized perspective reinforcement reinforcement learning text transformer type visual