Cross-Attention Fusion of Visual and Geometric Features for Large Vocabulary Arabic Lipreading

arXiv:2402.11520v1 Announce Type: new
Abstract: Lipreading involves using visual data to recognize spoken words by analyzing the movements of the lips and surrounding area. It is a hot research topic with many potential applications, such as human-machine interaction and enhancing audio speech recognition. Recent deep-learning based works aim to integrate visual features extracted from the mouth region with landmark points on the lip contours. However, employing a simple combination method such as concatenation may not be the most effective approach …

arabic arxiv attention cs.cv cs.mm features fusion type visual