all AI news
Compressing Transformer-based self-supervised models for speech processing. (arXiv:2211.09949v1 [cs.CL])
Nov. 21, 2022, 2:11 a.m. | Tzu-Quan Lin, Tsung-Huan Yang, Chun-Yao Chang, Kuang-Ming Chen, Tzu-hsun Feng, Hung-yi Lee, Hao Tang
cs.LG updates on arXiv.org arxiv.org
Despite the success of Transformers in self-supervised learning with
applications to various downstream tasks, the computational cost of training
and inference remains a major challenge for applying these models to a wide
spectrum of devices. Several isolated attempts have been made to compress
Transformers, prior to applying them to downstream tasks. In this work, we aim
to provide context for the isolated results, studying several commonly used
compression techniques, including weight pruning, head pruning, low-rank
approximation, and knowledge distillation. We …
More from arxiv.org / cs.LG updates on arXiv.org
Jobs in AI, ML, Big Data
Lead Developer (AI)
@ Cere Network | San Francisco, US
Research Engineer
@ Allora Labs | Remote
Ecosystem Manager
@ Allora Labs | Remote
Founding AI Engineer, Agents
@ Occam AI | New York
AI Engineer Intern, Agents
@ Occam AI | US
AI Research Scientist
@ Vara | Berlin, Germany and Remote