Similarity Guided Multimodal Fusion Transformer for Semantic Location Prediction in Social Media

arXiv:2405.05760v1 Announce Type: new
Abstract: The purpose of semantic location prediction is to extract relevant semantic location information from multimodal social media posts, offering a more contextual understanding of daily activities compared to GPS coordinates. However, this task becomes challenging due to the presence of noise and irrelevant information in "text-image" pairs. Existing methods suffer from insufficient feature representations and fail to consider the comprehensive integration of similarity at different granularities, making it difficult to filter out noise and irrelevant …

abstract arxiv cs.cl cs.cv daily extract fusion gps however information location media multimodal noise prediction semantic social social media transformer type understanding