Web: http://arxiv.org/abs/2201.08821

Jan. 24, 2022, 2:10 a.m. | Zhanghao Wu, Paras Jain, Matthew A. Wright, Azalia Mirhoseini, Joseph E. Gonzalez, Ion Stoica

cs.LG updates on arXiv.org arxiv.org

Graph neural networks are powerful architectures for structured datasets.
However, current methods struggle to represent long-range dependencies. Scaling
the depth or width of GNNs is insufficient to broaden receptive fields as
larger GNNs encounter optimization instabilities such as vanishing gradients
and representation oversmoothing, while pooling-based approaches have yet to
become as universally useful as in computer vision. In this work, we propose
the use of Transformer-based self-attention to learn long-range pairwise
relationships, with a novel "readout" mechanism to obtain a …

arxiv attention global global attention graph graph neural networks networks neural neural networks

More from arxiv.org / cs.LG updates on arXiv.org

Senior Data Engineer

@ DAZN | Hammersmith, London, United Kingdom

Sr. Data Engineer, Growth

@ Netflix | Remote, United States

Data Engineer - Remote

@ Craft | Wrocław, Lower Silesian Voivodeship, Poland

Manager, Operations Data Science

@ Binance.US | Vancouver

Senior Machine Learning Researcher for Copilot

@ GitHub | Remote - Europe

Sr. Marketing Data Analyst

@ HoneyBook | San Francisco, CA