Deep Neural Network approaches for Analysing Videos of Music Performances. (arXiv:2205.11232v2 [cs.CV] UPDATED)

This paper presents a framework to automate the labelling process for
gestures in musical performance videos with a 3D Convolutional Neural Network
(CNN). While this idea was proposed in a previous study, this paper introduces
several novelties: (i) Presents a novel method to overcome the class imbalance
challenge and make learning possible for co-existent gestures by batch
balancing approach and spatial-temporal representations of gestures. (ii)
Performs a detailed study on 7 and 18 categories of gestures generated during
the performance …

arxiv cv deep neural network music network neural network videos