Unsupervised active speaker detection in media content using cross-modal information. (arXiv:2209.11896v1 [eess.IV])

We present a cross-modal unsupervised framework for active speaker detection
in media content such as TV shows and movies. Machine learning advances have
enabled impressive performance in identifying individuals from speech and
facial images. We leverage speaker identity information from speech and faces,
and formulate active speaker detection as a speech-face assignment task such
that the active speaker's face and the underlying speech identify the same
person (character). We express the speech segments in terms of their associated
speaker identity …

arxiv detection information media unsupervised