Towards Learning to Imitate from a Single Video Demonstration

Agents that can learn to imitate behaviours observed in video -- without having direct access to internal state or action information of the observed agent -- are more suitable for learning in the natural world. However, formulating a reinforcement learning (RL) agent that facilitates this goal remains a significant challenge. We approach this challenge using contrastive training to learn a reward function by comparing an agent's behaviour with a single demonstration. We use a Siamese recurrent neural network architecture to …

agents architecture challenge function information learn natural network network architecture neural network recurrent neural network reinforcement reinforcement learning space state training video world