GTC: GNN-Transformer Co-contrastive Learning for Self-supervised Heterogeneous Graph Representation

arXiv:2403.15520v1 Announce Type: new
Abstract: Graph Neural Networks (GNNs) have emerged as the most powerful weapon for various graph tasks due to the message-passing mechanism's great local information aggregation ability. However, over-smoothing has always hindered GNNs from going deeper and capturing multi-hop neighbors. Unlike GNNs, Transformers can model global information and multi-hop interactions via multi-head self-attention and a proper Transformer structure can show more immunity to the over-smoothing problem. So, can we propose a novel framework to combine GNN and …

arxiv cs.ir cs.lg gnn graph graph representation gtc representation transformer type