Getting BART to Ride the Idiomatic Train: Learning to Represent Idiomatic Expressions. (arXiv:2207.03679v1 [cs.CL])

Idiomatic expressions (IEs), characterized by their non-compositionality, are
an important part of natural language. They have been a classical challenge to
NLP, including pre-trained language models that drive today's state-of-the-art.
Prior work has identified deficiencies in their contextualized representation
stemming from the underlying compositional paradigm of representation. In this
work, we take a first-principles approach to build idiomaticity into BART using
an adapter as a lightweight non-compositional language expert trained on
idiomatic sentences. The improved capability over baselines (e.g., BART) …

arxiv bart learning