Non-Intrusive Speech Intelligibility Prediction for Hearing Aids using Whisper and Metadata

arXiv:2309.09548v2 Announce Type: replace-cross
Abstract: Automated speech intelligibility assessment is pivotal for hearing aid (HA) development. In this paper, we present three novel methods to improve intelligibility prediction accuracy and introduce MBI-Net+, an enhanced version of MBI-Net, the top-performing system in the 1st Clarity Prediction Challenge. MBI-Net+ leverages Whisper's embeddings to create cross-domain acoustic features and includes metadata from speech signals by using a classifier that distinguishes different enhancement methods. Furthermore, MBI-Net+ integrates the hearing-aid speech perception index (HASPI) as …

abstract accuracy arxiv assessment automated challenge cs.lg cs.sd development eess.as hearing metadata novel paper pivotal prediction replace speech type whisper