Denoising-Diffusion Alignment for Continuous Sign Language Recognition

As a key to social good, continuous sign language recognition (CSLR) aims to promote active and accessible communication for the hearing impaired. Current CSLR research adopts a cross-modality alignment scheme to learn the mapping relationship between "video clip-textual gloss". However, this local alignment method, especially with weak data annotation, ignores the contextual information of modalities and directly reduces the generalization of visual features. To this end, we propose a novel Denoising-Diffusion global Alignment scheme (DDA), which focuses on modeling the …

alignment annotation clip communication continuous cs.cv current data data annotation denoising diffusion good hearing information key language learn mapping promote recognition relationship research social textual video video clip