all AI news
A Study on the Integration of Pre-trained SSL, ASR, LM and SLU Models for Spoken Language Understanding. (arXiv:2211.05869v1 [cs.CL])
cs.CL updates on arXiv.org arxiv.org
Collecting sufficient labeled data for spoken language understanding (SLU) is
expensive and time-consuming. Recent studies achieved promising results by
using pre-trained models in low-resource scenarios. Inspired by this, we aim to
ask: which (if any) pre-training strategies can improve performance across SLU
benchmarks? To answer this question, we employ four types of pre-trained models
and their combinations for SLU. We leverage self-supervised speech and language
models (LM) pre-trained on large quantities of unpaired data to extract strong
speech and text …
arxiv asr integration language slu spoken language understanding ssl study understanding