all AI news
Heterformer: A Transformer Architecture for Node Representation Learning on Heterogeneous Text-Rich Networks. (arXiv:2205.10282v1 [cs.CL])
cs.CL updates on arXiv.org arxiv.org
We study node representation learning on heterogeneous text-rich networks,
where nodes and edges are multi-typed and some types of nodes are associated
with text information. Although recent studies on graph neural networks (GNNs)
and pretrained language models (PLMs) have demonstrated their power in encoding
network and text signals, respectively, less focus has been given to delicately
coupling these two types of models on heterogeneous text-rich networks.
Specifically, existing GNNs rarely model text in each node in a contextualized
way; existing …
architecture arxiv learning networks node representation representation learning text transformer transformer architecture