Learning Separable Hidden Unit Contributions for Speaker-Adaptive Lip-Reading

arXiv:2310.05058v3 Announce Type: replace
Abstract: In this paper, we propose a novel method for speaker adaptation in lip reading, motivated by two observations. Firstly, a speaker's own characteristics can always be portrayed well by his/her few facial images or even a single image with shallow networks, while the fine-grained dynamic features associated with speech content expressed by the talking face always need deep sequential networks to represent accurately. Therefore, we treat the shallow and deep layers differently for speaker adaptive …

abstract arxiv cs.ai cs.cv cs.sd dynamic eess.as features fine-grained her hidden image images lip reading networks novel paper reading speaker type while