N-Modal Contrastive Losses with Applications to Social Media Data in Trimodal Space

arXiv:2403.12747v1 Announce Type: new
Abstract: The social media landscape of conflict dynamics has grown increasingly multi-modal. Recent advancements in model architectures such as CLIP have enabled researchers to begin studying the interplay between the modalities of text and images in a shared latent space. However, CLIP models fail to handle situations on social media when modalities present in a post expand above two. Social media dynamics often require understanding the interplay between not only text and images, but video as …

abstract applications architectures arxiv clip conflict cs.cv data dynamics however images landscape losses media media data modal multi-modal researchers social social media space studying text type