MSI-NeRF: Linking Omni-Depth with View Synthesis through Multi-Sphere Image aided Generalizable Neural Radiance Field

arXiv:2403.10840v1 Announce Type: cross
Abstract: Panoramic observation using fisheye cameras is significant in robot perception, reconstruction, and remote operation. However, panoramic images synthesized by traditional methods lack depth information and can only provide three degrees-of-freedom (3DoF) rotation rendering in virtual reality applications. To fully preserve and exploit the parallax information within the original fisheye cameras, we introduce MSI-NeRF, which combines deep learning omnidirectional depth estimation and novel view rendering. We first construct a multi-sphere image as a cost volume through …

abstract applications arxiv cameras cs.cv cs.ro freedom however image images information nerf neural radiance field observation perception reality rendering robot robot perception rotation sphere synthesis synthesized through type view virtual virtual reality