PromptBERT: Improving BERT Sentence Embeddings with Prompts. (arXiv:2201.04337v1 [cs.CL])

The poor performance of the original BERT for sentence semantic similarity
has been widely discussed in previous works. We find that unsatisfactory
performance is mainly due to the static token embeddings biases and the
ineffective BERT layers, rather than the high cosine similarity of the sentence
embeddings. To this end, we propose a prompt based sentence embeddings method
which can reduce token embeddings biases and make the original BERT layers more
effective. By reformulating the sentence embeddings task as the …

arxiv bert