NeRF-MAE : Masked AutoEncoders for Self Supervised 3D representation Learning for Neural Radiance Fields

arXiv:2404.01300v1 Announce Type: cross
Abstract: Neural fields excel in computer vision and robotics due to their ability to understand the 3D visual world such as inferring semantics, geometry, and dynamics. Given the capabilities of neural fields in densely representing a 3D scene from 2D images, we ask the question: Can we scale their self-supervised pretraining, specifically using masked autoencoders, to generate effective 3D representations from posed RGB images. Owing to the astounding success of extending transformers to novel data modalities, …

abstract arxiv autoencoders capabilities computer computer vision cs.ai cs.cv cs.lg dynamics excel fields geometry images nerf neural radiance fields representation representation learning robotics semantics type vision visual world