Masked Autoencoders As Spatiotemporal Learners. (arXiv:2205.09113v1 [cs.CV])

This paper studies a conceptually simple extension of Masked Autoencoders
(MAE) to spatiotemporal representation learning from videos. We randomly mask
out spacetime patches in videos and learn an autoencoder to reconstruct them in
pixels. Interestingly, we show that our MAE method can learn strong
representations with almost no inductive bias on spacetime (only except for
patch and positional embeddings), and spacetime-agnostic random masking
performs the best. We observe that the optimal masking ratio is as high as 90%
(vs. 75% …

arxiv cv