Comparing Graph Transformers via Positional Encodings

arXiv:2402.14202v1 Announce Type: new
Abstract: The distinguishing power of graph transformers is closely tied to the choice of positional encoding: features used to augment the base transformer with information about the graph. There are two primary types of positional encoding: absolute positional encodings (APEs) and relative positional encodings (RPEs). APEs assign features to each node and are given as input to the transformer. RPEs instead assign a feature to each pair of nodes, e.g., graph distance, and are used to …

abstract arxiv cs.lg encoding features graph information positional encoding power transformer transformers type types via