[Own work] MM-SHAP to measure modality contributions

Today we present our own work on MM-SHAP which measures how much a multimodal model uses each modality. Ah, what is multimodality again? 👉 https://youtu.be/jReaoJWdO78

📜 Parcalabescu, Letitia, and Anette Frank. "MM-SHAP: A Performance-agnostic Metric for Measuring Multimodal Contributions in Vision and Language Models & Tasks." arXiv preprint arXiv:2212.08158 (2022). https://arxiv.org/abs/2212.08158

📺 VeLO trained optimizers: https://youtu.be/9a6PQJxzUpM
📺 Watermarking Large Language models: https://youtu.be/-vToUx5SDW4
📺 Paella text-to-image diffusion model: https://youtu.be/6zeLSANd41k

❓Check out our #MachineLearning Quiz Questions: https://www.youtube.com/c/AICoffeeBreak/community

Outline:
00:00 Paper for ACL …

acl information language paper shap support toronto transformers vision words