Polynormer: Polynomial-Expressive Graph Transformer in Linear Time

arXiv:2403.01232v1 Announce Type: new
Abstract: Graph transformers (GTs) have emerged as a promising architecture that is theoretically more expressive than message-passing graph neural networks (GNNs). However, typical GT models have at least quadratic complexity and thus cannot scale to large graphs. While there are several linear GTs recently proposed, they still lag behind GNN counterparts on several popular graph datasets, which poses a critical concern on their practical expressivity. To balance the trade-off between expressivity and scalability of GTs, we …

abstract architecture arxiv complexity cs.ai cs.lg gnns graph graph neural networks graphs least linear networks neural networks polynomial scale transformer transformers type