A Unified Framework for Rank-based Evaluation Metrics for Link Prediction in Knowledge Graphs. (arXiv:2203.07544v2 [cs.LG] UPDATED)

The link prediction task on knowledge graphs without explicit negative
triples in the training data motivates the usage of rank-based metrics. Here,
we review existing rank-based metrics and propose desiderata for improved
metrics to address lack of interpretability and comparability of existing
metrics to datasets of different sizes and properties. We introduce a simple
theoretical framework for rank-based metrics upon which we investigate two
avenues for improvements to existing metrics via alternative aggregation
functions and concepts from probability theory. We …

arxiv evaluation framework graphs knowledge knowledge graphs link prediction metrics prediction