Self-Supervised Keypoint Discovery in Behavioral Videos. (arXiv:2112.05121v2 [cs.CV] UPDATED)

We propose a method for learning the posture and structure of agents from
unlabelled behavioral videos. Starting from the observation that behaving
agents are generally the main sources of movement in behavioral videos, our
method, Behavioral Keypoint Discovery (B-KinD), uses an encoder-decoder
architecture with a geometric bottleneck to reconstruct the spatiotemporal
difference between video frames. By focusing only on regions of movement, our
approach works directly on input videos without requiring manual annotations.
Experiments on a variety of agent types …

arxiv cv discovery videos