Aria-NeRF: Multimodal Egocentric View Synthesis

arXiv:2311.06455v2 Announce Type: replace
Abstract: We seek to accelerate research in developing rich, multimodal scene models trained from egocentric data, based on differentiable volumetric ray-tracing inspired by Neural Radiance Fields (NeRFs). The construction of a NeRF-like model from an egocentric image sequence plays a pivotal role in understanding human behavior and holds diverse applications within the realms of VR/AR. Such egocentric NeRF-like models may be used as realistic simulations, contributing significantly to the advancement of intelligent agents capable of executing …

abstract applications aria arxiv behavior construction cs.ai cs.cv data differentiable diverse diverse applications fields human image multimodal nerf neural radiance fields pivotal ray research role synthesis tracing type understanding view