Web: http://arxiv.org/abs/2109.03264

May 11, 2022, 1:11 a.m. | Eugene Kharitonov, Ann Lee, Adam Polyak, Yossi Adi, Jade Copet, Kushal Lakhotia, Tu-Anh Nguyen, Morgane Rivière, Abdelrahman Mohamed, Emmanuel Du

cs.LG updates on arXiv.org arxiv.org

Speech pre-training has primarily demonstrated efficacy on classification
tasks, while its capability of generating novel speech, similar to how GPT-2
can generate coherent paragraphs, has barely been explored. Generative Spoken
Language Modeling (GSLM) \cite{Lakhotia2021} is the only prior work addressing
the generative aspects of speech pre-training, which replaces text with
discovered phone-like units for language modeling and shows the ability to
generate meaningful novel sentences. Unfortunately, despite eliminating the
need of text, the units used in GSLM discard most of …

arxiv free language modeling text

More from arxiv.org / cs.LG updates on arXiv.org

Director, Applied Mathematics & Computational Research Division

@ Lawrence Berkeley National Lab | Berkeley, Ca

Business Data Analyst

@ MainStreet Family Care | Birmingham, AL

Assistant/Associate Professor of the Practice in Business Analytics

@ Georgetown University McDonough School of Business | Washington DC

Senior Data Science Writer

@ NannyML | Remote

Director of AI/ML Engineering

@ Armis Industries | Remote (US only), St. Louis, California

Digital Analytics Manager

@ Patagonia | Ventura, California