Hate-CLIPper: Multimodal Hateful Meme Classification based on Cross-modal Interaction of CLIP Features. (arXiv:2210.05916v2 [cs.CL] UPDATED)

Hateful memes are a growing menace on social media. While the image and its
corresponding text in a meme are related, they do not necessarily convey the
same meaning when viewed individually. Hence, detecting hateful memes requires
careful consideration of both visual and textual information. Multimodal
pre-training can be beneficial for this task because it effectively captures
the relationship between the image and the text by representing them in a
similar feature space. Furthermore, it is essential to model the …

arxiv classification clip features meme multimodal