Deliberation Model for On-Device Spoken Language Understanding. (arXiv:2204.01893v3 [cs.CL] UPDATED)

We propose a novel deliberation-based approach to end-to-end (E2E) spoken
language understanding (SLU), where a streaming automatic speech recognition
(ASR) model produces the first-pass hypothesis and a second-pass natural
language understanding (NLU) component generates the semantic parse by
conditioning on both ASR's text and audio embeddings. By formulating E2E SLU as
a generalized decoder, our system is able to support complex compositional
semantic structures. Furthermore, the sharing of parameters between ASR and NLU
makes the system especially suitable for resource-constrained …

arxiv language spoken language understanding understanding