Incorporating simulated spatial context information improves the effectiveness of contrastive learning models

arXiv:2401.15120v2 Announce Type: replace
Abstract: Visual learning often occurs in a specific context, where an agent acquires skills through exploration and tracking of its location in a consistent environment. The historical spatial context of the agent provides a similarity signal for self-supervised contrastive learning. We present a unique approach, termed Environmental Spatial Similarity (ESS), that complements existing contrastive learning methods. Using images from simulated, photorealistic environments as an experimental setting, we demonstrate that ESS outperforms traditional instance discrimination approaches. Moreover, …

abstract agent arxiv consistent context cs.ai cs.cv environment exploration information location signal skills spatial through tracking type visual