Walkthrough of Graph Attention Network (GAT) with Visualized Implementation

Understanding Graph Neural Networks (GNNs) is increasingly relevant as transformers continue tackling graph problems like from the Open Graph Benchmark. Even if natural language is all a graph needs, GNNs remain a fruitful source of inspiration for future methods.

In this post, I’m going to walk through the implementation of a vanilla GNN layer, and then show the modifications for a vanilla graph attention layer as described by the ICLR paper titled Graph …

machine learning transformers