GNNBENCH: Fair and Productive Benchmarking for Single-GPU GNN System

arXiv:2404.04118v1 Announce Type: new
Abstract: We hypothesize that the absence of a standardized benchmark has allowed several fundamental pitfalls in GNN System design and evaluation that the community has overlooked. In this work, we propose GNNBench, a plug-and-play benchmarking platform focused on system innovation. GNNBench presents a new protocol to exchange their captive tensor data, supports custom classes in System APIs, and allows automatic integration of the same system module to many deep learning frameworks, such as PyTorch and TensorFlow. …

abstract arxiv benchmark benchmarking community cs.dc cs.lg design evaluation fair gnn gpu innovation platform productive protocol type work