Enhancing Graph Transformers with Hierarchical Distance Structural Encoding

Graph transformers need strong inductive biases to derive meaningful attention scores. Yet, current methods often fall short in capturing longer ranges, hierarchical structures, or community structures, which are common in various graphs such as molecules, social networks, and citation networks. This paper presents a Hierarchical Distance Structural Encoding (HDSE) method to model node distances in a graph, focusing on its multi-level, hierarchical nature. We introduce a novel framework to seamlessly integrate HDSE into the attention mechanism of existing graph transformers, …

attention biases community cs.ai cs.lg cs.si current encoding graph graphs hierarchical inductive molecules networks paper social social networks transformers