Towards Online Real-Time Memory-based Video Inpainting Transformers

arXiv:2403.16161v1 Announce Type: new
Abstract: Video inpainting tasks have seen significant improvements in recent years with the rise of deep neural networks and, in particular, vision transformers. Although these models show promising reconstruction quality and temporal consistency, they are still unsuitable for live videos, one of the last steps to make them completely convincing and usable. The main limitations are that these state-of-the-art models inpaint using the whole video (offline processing) and show an insufficient frame rate. In our approach, …

abstract arxiv cs.cv improvements inpainting memory networks neural networks quality real-time show tasks temporal them transformers type video videos vision vision transformers