MetricBERT: Text Representation Learning via Self-Supervised Triplet Training. (arXiv:2208.06610v1 [cs.CL])

We present MetricBERT, a BERT-based model that learns to embed text under a
well-defined similarity metric while simultaneously adhering to the
``traditional'' masked-language task. We focus on downstream tasks of learning
similarities for recommendations where we show that MetricBERT outperforms
state-of-the-art alternatives, sometimes by a substantial margin. We conduct
extensive evaluations of our method and its different variants, showing that
our training objective is highly beneficial over a traditional contrastive
loss, a standard cosine similarity objective, and six other baselines. …

arxiv learning representation representation learning text training