Flaky Performances when Pretraining on Relational Databases. (arXiv:2211.05213v1 [cs.LG])

We explore the downstream task performances for graph neural network (GNN)
self-supervised learning (SSL) methods trained on subgraphs extracted from
relational databases (RDBs). Intuitively, this joint use of SSL and GNNs should
allow to leverage more of the available data, which could translate to better
results. However, we found that naively porting contrastive SSL techniques can
cause ``negative transfer'': linear evaluation on fixed representations from a
pretrained model performs worse than on representations from the
randomly-initialized model. Based on the …

arxiv databases relational databases