Web: http://arxiv.org/abs/2110.02802

Jan. 31, 2022, 2:10 a.m. | Xavier Suau, Luca Zappella, Nicholas Apostoloff

cs.CL updates on arXiv.org arxiv.org

In this paper we aim to investigate the mechanisms that guide text generation
with pre-trained Transformer-based Language Models (TLMs). Grounded on the
Product of Experts formulation by Hinton (1999), we describe a generative
mechanism that exploits expert units which naturally exist in TLMs. Such units
are responsible for detecting concepts in the input and conditioning text
generation on such concepts. We describe how to identify expert units and how
to activate them during inference in order to induce any desired …

arxiv language language models models

More from arxiv.org / cs.CL updates on arXiv.org

Data Scientist

@ Fluent, LLC | Boca Raton, Florida, United States

Big Data ETL Engineer

@ Binance.US | Vancouver

Data Scientist / Data Engineer

@ Kin + Carta | Chicago

Data Engineer

@ Craft | Warsaw, Masovian Voivodeship, Poland

Senior Manager, Data Analytics Audit

@ Affirm | Remote US

Data Scientist - Nationwide Opportunities, AWS Professional Services

@ Amazon.com | US, NC, Virtual Location - N Carolina