Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFinger. (arXiv:2108.09779v2 [cs.RO] UPDATED)

We present a system for learning a challenging dexterous manipulation task
involving moving a cube to an arbitrary 6-DoF pose with only 3-fingers trained
with NVIDIA's IsaacGym simulator. We show empirical benefits, both in
simulation and sim-to-real transfer, of using keypoints as opposed to
position+quaternion representations for the object pose in 6-DoF for policy
observations and in reward calculation to train a model-free reinforcement
learning agent. By utilizing domain randomization strategies along with the
keypoint representation of the pose of …

arxiv gpu remote simulation