SGHormer: An Energy-Saving Graph Transformer Driven by Spikes

arXiv:2403.17656v1 Announce Type: cross
Abstract: Graph Transformers (GTs) with powerful representation learning ability make a huge success in wide range of graph tasks. However, the costs behind outstanding performances of GTs are higher energy consumption and computational overhead. The complex structure and quadratic complexity during attention calculation in vanilla transformer seriously hinder its scalability on the large-scale graph data. Though existing methods have made strides in simplifying combinations among blocks or attention-learning paradigm to improve GTs' efficiency, a series of …

abstract arxiv attention complexity computational consumption costs cs.ai cs.lg cs.ne energy graph hinder however performances representation representation learning saving success tasks transformer transformers type vanilla transformer