MIDGET: Music Conditioned 3D Dance Generation

arXiv:2404.12062v1 Announce Type: cross
Abstract: In this paper, we introduce a MusIc conditioned 3D Dance GEneraTion model, named MIDGET based on Dance motion Vector Quantised Variational AutoEncoder (VQ-VAE) model and Motion Generative Pre-Training (GPT) model to generate vibrant and highquality dances that match the music rhythm. To tackle challenges in the field, we introduce three new components: 1) a pre-trained memory codebook based on the Motion VQ-VAE model to store different human pose codes, 2) employing Motion GPT model to …

abstract arxiv autoencoder challenges cs.cv cs.gr cs.sd dance eess.as generate generative gpt match music paper pre-training training type vae vector