A Simple Approach for Visual Rearrangement: 3D Mapping and Semantic Search. (arXiv:2206.13396v2 [cs.CV] UPDATED)

Physically rearranging objects is an important capability for embodied
agents. Visual room rearrangement evaluates an agent's ability to rearrange
objects in a room to a desired goal based solely on visual input. We propose a
simple yet effective method for this problem: (1) search for and map which
objects need to be rearranged, and (2) rearrange each object until the task is
complete. Our approach consists of an off-the-shelf semantic segmentation
model, voxel-based semantic map, and semantic search policy to …

3d 3d mapping arxiv cv mapping search semantic