PaECTER: Patent-level Representation Learning using Citation-informed Transformers

arXiv:2402.19411v1 Announce Type: cross
Abstract: PaECTER is a publicly available, open-source document-level encoder specific for patents. We fine-tune BERT for Patents with examiner-added citation information to generate numerical representations for patent documents. PaECTER performs better in similarity tasks than current state-of-the-art models used in the patent domain. More specifically, our model outperforms the next-best patent specific pre-trained language model (BERT for Patents) on our patent citation prediction test dataset on two different rank evaluation metrics. PaECTER predicts at least one …

abstract art arxiv bert cs.cl cs.ir cs.lg current document documents domain encoder generate information numerical patent patents representation representation learning state state-of-the-art models tasks transformers type