AD3: Implicit Action is the Key for World Models to Distinguish the Diverse Visual Distractors

arXiv:2403.09976v1 Announce Type: new
Abstract: Model-based methods have significantly contributed to distinguishing task-irrelevant distractors for visual control. However, prior research has primarily focused on heterogeneous distractors like noisy background videos, leaving homogeneous distractors that closely resemble controllable agents largely unexplored, which poses significant challenges to existing methods. To tackle this problem, we propose Implicit Action Generator (IAG) to learn the implicit actions of visual distractors, and present a new algorithm named implicit Action-informed Diverse visual Distractors Distinguisher (AD3), that leverages …

abstract agents arxiv challenges contributed control cs.cv cs.lg diverse however key prior research resemble the key type videos visual world world models