Enhancing Bandwidth Efficiency for Video Motion Transfer Applications using Deep Learning Based Keypoint Prediction

arXiv:2403.11337v1 Announce Type: new
Abstract: We propose a deep learning based novel prediction framework for enhanced bandwidth reduction in motion transfer enabled video applications such as video conferencing, virtual reality gaming and privacy preservation for patient health monitoring. To model complex motion, we use the First Order Motion Model (FOMM) that represents dynamic objects using learned keypoints along with their local affine transformations. Keypoints are extracted by a self-supervised keypoint detector and organized in a time series corresponding to the …

abstract applications arxiv bandwidth conferencing cs.ai cs.cv deep learning efficiency framework gaming health monitoring novel patient prediction preservation privacy reality transfer type video video conferencing virtual virtual reality