Improving Contextual Congruence Across Modalities for Effective Multimodal Marketing using Knowledge-infused Learning

The prevalence of smart devices with the ability to capture moments in multiple modalities has enabled users to experience multimodal information online. However, large Language (LLMs) and Vision models (LVMs) are still limited in capturing holistic meaning with cross-modal semantic relationships. Without explicit, common sense knowledge (e.g., as a knowledge graph), Visual Language Models (VLMs) only learn implicit representations by capturing high-level patterns in vast corpora, missing essential contextual cross-modal cues. In this work, we design a framework to couple …

common sense cs.ai cs.cl cs.cv cs.cy cs.hc devices experience information knowledge language large language llms marketing meaning modal moments multimodal multiple relationships semantic sense smart vision vision models