FreshGNN: Reducing Memory Access via Stable Historical Embeddings for Graph Neural Network Training

arXiv:2301.07482v3 Announce Type: replace
Abstract: A key performance bottleneck when training graph neural network (GNN) models on large, real-world graphs is loading node features onto a GPU. Due to limited GPU memory, expensive data movement is necessary to facilitate the storage of these features on alternative devices with slower access (e.g. CPU memory). Moreover, the irregularity of graph structures contributes to poor data locality which further exacerbates the problem. Consequently, existing frameworks capable of efficiently training large GNN models usually …

abstract arxiv cs.lg data data movement devices embeddings features gnn gpu graph graph neural network graphs key loading memory network network training neural network node performance storage training type via world