LP-BERT: Multi-task Pre-training Knowledge Graph BERT for Link Prediction. (arXiv:2201.04843v1 [cs.CL])

Link prediction plays an significant role in knowledge graph, which is an
important resource for many artificial intelligence tasks, but it is often
limited by incompleteness. In this paper, we propose knowledge graph BERT for
link prediction, named LP-BERT, which contains two training stages: multi-task
pre-training and knowledge graph fine-tuning. The pre-training strategy not
only uses Mask Language Model (MLM) to learn the knowledge of context corpus,
but also introduces Mask Entity Model (MEM) and Mask Relation Model (MRM),
which …

arxiv bert graph knowledge graph link prediction prediction training