Large Language Models, MirrorBERT — Transforming Models into Universal Lexical and Sentence…

Large Language Models, MirrorBERT — Transforming Models into Universal Lexical and Sentence Encoders

Discover how mirror augmentation generates data and aces the BERT performance on semantic similarity tasks

Introduction

It is no secret that BERT-like models play a fundamental role in modern NLP applications. Despite their phenomenal performance on downstream tasks, most of these models are not that perfect on specific problems without fine-tuning. Embedding construction from raw pretrained models usually leads to metrics being far from state-of-the-art results. At …

applications augmentation bert contrastive-learning data language language models large language large language models machine learning modern nlp performance role secret semantic tasks transformers