all AI news
SKILL: Similarity-aware Knowledge distILLation for Speech Self-Supervised Learning
Feb. 27, 2024, 5:50 a.m. | Luca Zampierin, Ghouthi Boukli Hacene, Bac Nguyen, Mirco Ravanelli
cs.CL updates on arXiv.org arxiv.org
Abstract: Self-supervised learning (SSL) has achieved remarkable success across various speech-processing tasks. To enhance its efficiency, previous works often leverage the use of compression techniques. A notable recent attempt is DPHuBERT, which applies joint knowledge distillation (KD) and structured pruning to learn a significantly smaller SSL model. In this paper, we contribute to this research domain by introducing SKILL, a novel method that conducts distillation across groups of layers instead of distilling individual arbitrarily selected layers …
abstract arxiv compression cs.cl cs.lg cs.sd distillation eess.as efficiency knowledge learn processing pruning self-supervised learning speech ssl success supervised learning tasks type
More from arxiv.org / cs.CL updates on arXiv.org
Jobs in AI, ML, Big Data
Founding AI Engineer, Agents
@ Occam AI | New York
AI Engineer Intern, Agents
@ Occam AI | US
AI Research Scientist
@ Vara | Berlin, Germany and Remote
Data Architect
@ University of Texas at Austin | Austin, TX
Data ETL Engineer
@ University of Texas at Austin | Austin, TX
Business Intelligence Architect - Specialist
@ Eastman | Hyderabad, IN, 500 008