Lester: rotoscope animation through video object segmentation and tracking

arXiv:2402.09883v1 Announce Type: new
Abstract: This article introduces Lester, a novel method to automatically synthetise retro-style 2D animations from videos. The method approaches the challenge mainly as an object segmentation and tracking problem. Video frames are processed with the Segment Anything Model (SAM) and the resulting masks are tracked through subsequent frames with DeAOT, a method of hierarchical propagation for semi-supervised video object segmentation. The geometry of the masks' contours is simplified with the Douglas-Peucker algorithm. Finally, facial traits, pixelation …

abstract animation animations article arxiv challenge cs.ai cs.cv cs.gr cs.mm masks novel sam segment segment anything segment anything model segmentation style through tracking type video videos