How Physics and Background Attributes Impact Video Transformers in Robotic Manipulation: A Case Study on Planar Pushing

arXiv:2310.02044v3 Announce Type: replace-cross
Abstract: As model and dataset sizes continue to scale in robot learning, the need to understand what is the specific factor in the dataset that affects model performance becomes increasingly urgent to ensure cost-effective data collection and model performance. In this work, we empirically investigate how physics attributes (color, friction coefficient, shape) and scene background characteristics, such as the complexity and dynamics of interactions with background objects, influence the performance of Video Transformers in predicting planar …

arxiv case case study cs.cv cs.ro impact manipulation physics robotic robotic manipulation study transformers type video