Put Myself in Your Shoes: Lifting the Egocentric Perspective from Exocentric Videos

arXiv:2403.06351v1 Announce Type: new
Abstract: We investigate exocentric-to-egocentric cross-view translation, which aims to generate a first-person (egocentric) view of an actor based on a video recording that captures the actor from a third-person (exocentric) perspective. To this end, we propose a generative framework called Exo2Ego that decouples the translation process into two stages: high-level structure transformation, which explicitly encourages cross-view correspondence between exocentric and egocentric views, and a diffusion-based pixel-level hallucination, which incorporates a hand layout prior to enhance the …

abstract actor arxiv cs.cv framework generate generative person perspective recording translation type video videos view