See, Imagine, Plan: Discovering and Hallucinating Tasks from a Single Image

arXiv:2403.13438v1 Announce Type: new
Abstract: Humans can not only recognize and understand the world in its current state but also envision future scenarios that extend beyond immediate perception. To resemble this profound human capacity, we introduce zero-shot task hallucination -- given a single RGB image of any scene comprising unknown environments and objects, our model can identify potential tasks and imagine their execution in a vivid narrative, realized as a video. We develop a modular pipeline that progressively enhances scene …

arxiv cs.cv image imagine tasks type