Delayed Bottlenecking: Alleviating Forgetting in Pre-trained Graph Neural Networks

arXiv:2404.14941v1 Announce Type: new
Abstract: Pre-training GNNs to extract transferable knowledge and apply it to downstream tasks has become the de facto standard of graph representation learning. Recent works focused on designing self-supervised pre-training tasks to extract useful and universal transferable knowledge from large-scale unlabeled data. However, they have to face an inevitable question: traditional pre-training strategies that aim at extracting useful information about pre-training tasks, may not extract all useful information about the downstream task. In this paper, we …

abstract apply arxiv become cs.ai cs.lg data designing extract face gnns graph graph neural networks graph representation however knowledge networks neural networks pre-training representation representation learning scale standard tasks training type universal