Landmark-Guided Cross-Speaker Lip Reading with Mutual Information Regularization

arXiv:2403.16071v1 Announce Type: cross
Abstract: Lip reading, the process of interpreting silent speech from visual lip movements, has gained rising attention for its wide range of realistic applications. Deep learning approaches greatly improve current lip reading systems. However, lip reading in cross-speaker scenarios where the speaker identity changes, poses a challenging problem due to inter-speaker variability. A well-trained lip reading system may perform poorly when handling a brand new speaker. To learn a speaker-robust lip reading model, a key insight …

abstract applications arxiv attention cs.ai cs.cv cs.mm current deep learning however identity information landmark lip reading movements process reading regularization speaker speech systems type visual